[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:06:18 dist-test-slave-69rg krb5kdc[28558](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:06:18 dist-test-slave-69rg krb5kdc[28558](info): set up 2 sockets
Aug 06 21:06:18 dist-test-slave-69rg krb5kdc[28558](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 21:06:19.240700 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.003s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:06:19 dist-test-slave-69rg krb5kdc[28558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:06:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:06:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:06:19.399256 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:42211
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:36021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:42211
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:19.545102 28571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:19.545493 28571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:19.545574 28571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:19.550328 28571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:06:19.550446 28571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:19.550483 28571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:19.550521 28571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:06:19.550562 28571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:06:19.557314 28571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:36021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:42211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:42211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:19.559073 28571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:19.560560 28571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:06:19.576581 28571 server_base.cc:1034] running on GCE node
W20260806 21:06:19.576754 28577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:19.576561 28576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:19.577625 28579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:19.578563 28571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:19.580474 28571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:19.581724 28571 hybrid_clock.cc:648] HybridClock initialized: now 1786050379581709 us; error 36 us; skew 500 ppm
Aug 06 21:06:19 dist-test-slave-69rg krb5kdc[28558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050379, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:19.585314 28571 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:06:19.586853 28571 webserver.cc:460] Webserver started at http://127.27.225.254:42599/ using document root <none> and password file <none>
I20260806 21:06:19.587905 28571 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:19.588024 28571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:19.588515 28571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:19.591667 28571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "24ba0d20815d45cbaec5871b8a1ffe4b"
format_stamp: "Formatted at 2026-08-06 21:06:19 on dist-test-slave-69rg"
server_key: "42aa04a93582ca62fc761120c1cee1af"
server_key_iv: "7361d38a4198597a3187201453afa0de"
server_key_version: "encryptionkey@0"
I20260806 21:06:19.592383 28571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "24ba0d20815d45cbaec5871b8a1ffe4b"
format_stamp: "Formatted at 2026-08-06 21:06:19 on dist-test-slave-69rg"
server_key: "42aa04a93582ca62fc761120c1cee1af"
server_key_iv: "7361d38a4198597a3187201453afa0de"
server_key_version: "encryptionkey@0"
I20260806 21:06:19.598807 28571 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 21:06:19.603086 28586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:19.605087 28571 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:06:19.605333 28571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "24ba0d20815d45cbaec5871b8a1ffe4b"
format_stamp: "Formatted at 2026-08-06 21:06:19 on dist-test-slave-69rg"
server_key: "42aa04a93582ca62fc761120c1cee1af"
server_key_iv: "7361d38a4198597a3187201453afa0de"
server_key_version: "encryptionkey@0"
I20260806 21:06:19.605474 28571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:19.622972 28571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:19.627009 28571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:19.627313 28571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:19.637799 28571 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:42211
I20260806 21:06:19.637962 28638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:42211 every 8 connection(s)
I20260806 21:06:19.639290 28571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:06:19.646919 28639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:19.649701 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 28571
I20260806 21:06:19.649951 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:06:19.651523 28551 external_mini_cluster.cc:1428] Setting key 68802e831fa8e048d65c3b0aebe4cb85
I20260806 21:06:19.657290 28639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24ba0d20815d45cbaec5871b8a1ffe4b: Bootstrap starting.
I20260806 21:06:19.663033 28639 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24ba0d20815d45cbaec5871b8a1ffe4b: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:19.664953 28639 log.cc:824] T 00000000000000000000000000000000 P 24ba0d20815d45cbaec5871b8a1ffe4b: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:19.669143 28639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24ba0d20815d45cbaec5871b8a1ffe4b: No bootstrap required, opened a new log
I20260806 21:06:19.675652 28639 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24ba0d20815d45cbaec5871b8a1ffe4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ba0d20815d45cbaec5871b8a1ffe4b" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42211 } }
I20260806 21:06:19.676043 28639 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24ba0d20815d45cbaec5871b8a1ffe4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:19.676168 28639 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24ba0d20815d45cbaec5871b8a1ffe4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24ba0d20815d45cbaec5871b8a1ffe4b, State: Initialized, Role: FOLLOWER
I20260806 21:06:19.677043 28639 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24ba0d20815d45cbaec5871b8a1ffe4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ba0d20815d45cbaec5871b8a1ffe4b" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42211 } }
I20260806 21:06:19.677295 28639 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24ba0d20815d45cbaec5871b8a1ffe4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:06:19.677444 28639 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24ba0d20815d45cbaec5871b8a1ffe4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:06:19.677610 28639 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24ba0d20815d45cbaec5871b8a1ffe4b [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:19.679041 28639 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24ba0d20815d45cbaec5871b8a1ffe4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ba0d20815d45cbaec5871b8a1ffe4b" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42211 } }
I20260806 21:06:19.679689 28639 leader_election.cc:302] T 00000000000000000000000000000000 P 24ba0d20815d45cbaec5871b8a1ffe4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24ba0d20815d45cbaec5871b8a1ffe4b; no voters:
I20260806 21:06:19.680224 28639 leader_election.cc:288] T 00000000000000000000000000000000 P 24ba0d20815d45cbaec5871b8a1ffe4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:06:19.680397 28644 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24ba0d20815d45cbaec5871b8a1ffe4b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:19.680665 28644 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24ba0d20815d45cbaec5871b8a1ffe4b [term 1 LEADER]: Becoming Leader. State: Replica: 24ba0d20815d45cbaec5871b8a1ffe4b, State: Running, Role: LEADER
Aug 06 21:06:19 dist-test-slave-69rg krb5kdc[28558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:19.681185 28644 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24ba0d20815d45cbaec5871b8a1ffe4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ba0d20815d45cbaec5871b8a1ffe4b" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42211 } }
I20260806 21:06:19.682219 28639 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24ba0d20815d45cbaec5871b8a1ffe4b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:06:19.684242 28646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24ba0d20815d45cbaec5871b8a1ffe4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24ba0d20815d45cbaec5871b8a1ffe4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ba0d20815d45cbaec5871b8a1ffe4b" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42211 } } }
I20260806 21:06:19.684399 28646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24ba0d20815d45cbaec5871b8a1ffe4b [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:19.684881 28645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24ba0d20815d45cbaec5871b8a1ffe4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24ba0d20815d45cbaec5871b8a1ffe4b. Latest consensus state: current_term: 1 leader_uuid: "24ba0d20815d45cbaec5871b8a1ffe4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24ba0d20815d45cbaec5871b8a1ffe4b" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42211 } } }
I20260806 21:06:19.685029 28645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24ba0d20815d45cbaec5871b8a1ffe4b [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:19.685428 28653 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:06:19.691448 28642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:19.660043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53816 (local address 127.27.225.254:42211)
0806 21:06:19.660728 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:19.660754 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:19.660835 (+ 81us) server_negotiation.cc:407] Connection header received
0806 21:06:19.667362 (+ 6527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:19.667388 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:19.667872 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:19.668234 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:19.672489 (+ 4255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:19.673622 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:19.675721 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:19.676437 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:19.676655 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:19.682143 (+ 5488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:19.682193 (+ 50us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:19.682211 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:19.682278 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:19.685545 (+ 3267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:19.686512 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:19.686520 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:19.686526 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:19.686697 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:19.687259 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:19.687263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:19.687265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:19.688047 (+ 782us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:19.688298 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:19.689293 (+ 995us) server_negotiation.cc:299] Negotiation successful
0806 21:06:19.689772 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":213,"threads_started":1}
I20260806 21:06:19.691771 28653 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:06:19.701480 28653 catalog_manager.cc:1347] Generated new cluster ID: b57abaf26e0b47a8b803f01cc64e4764
I20260806 21:06:19.701597 28653 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:06:19.725118 28653 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:06:19.726377 28653 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:06:19.732137 28653 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24ba0d20815d45cbaec5871b8a1ffe4b: Generated new TSK 0
I20260806 21:06:19.733188 28653 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.225.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
I20260806 21:06:19.771500 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:42211
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:36021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:19.906452 28665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:19.906960 28665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:19.907089 28665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:19.911840 28665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:19.911947 28665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:19.912048 28665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:06:19.917996 28665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:36021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:42211
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:19.919273 28665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:19.920359 28665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:19.931449 28670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:19.931496 28673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:19.931545 28671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:19.931850 28665 server_base.cc:1034] running on GCE node
I20260806 21:06:19.932334 28665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:19.933053 28665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:19.934237 28665 hybrid_clock.cc:648] HybridClock initialized: now 1786050379934214 us; error 33 us; skew 500 ppm
Aug 06 21:06:19 dist-test-slave-69rg krb5kdc[28558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050379, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:19.937140 28665 init.cc:377] Logged in from keytab as kudu/127.27.225.193@KRBTEST.COM (short username kudu)
I20260806 21:06:19.938248 28665 webserver.cc:460] Webserver started at http://127.27.225.193:37287/ using document root <none> and password file <none>
I20260806 21:06:19.938810 28665 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:19.938872 28665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:19.939045 28665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:19.941124 28665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "2355df5ef8784148b15df257c39ba222"
format_stamp: "Formatted at 2026-08-06 21:06:19 on dist-test-slave-69rg"
server_key: "7aa911aec74bf30c4496c92d133fd439"
server_key_iv: "422c2ce8fb316e9a53ffcf0ff0f58d94"
server_key_version: "encryptionkey@0"
I20260806 21:06:19.941686 28665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "2355df5ef8784148b15df257c39ba222"
format_stamp: "Formatted at 2026-08-06 21:06:19 on dist-test-slave-69rg"
server_key: "7aa911aec74bf30c4496c92d133fd439"
server_key_iv: "422c2ce8fb316e9a53ffcf0ff0f58d94"
server_key_version: "encryptionkey@0"
I20260806 21:06:19.945617 28665 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260806 21:06:19.948266 28680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:19.949527 28665 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:06:19.949633 28665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "2355df5ef8784148b15df257c39ba222"
format_stamp: "Formatted at 2026-08-06 21:06:19 on dist-test-slave-69rg"
server_key: "7aa911aec74bf30c4496c92d133fd439"
server_key_iv: "422c2ce8fb316e9a53ffcf0ff0f58d94"
server_key_version: "encryptionkey@0"
I20260806 21:06:19.949767 28665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:19.989558 28665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:19.993361 28665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:19.993625 28665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:19.994467 28665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:19.995746 28665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:19.995800 28665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:19.995844 28665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:19.995899 28665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:20.009550 28793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:39955 every 8 connection(s)
I20260806 21:06:20.009519 28665 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:39955
I20260806 21:06:20.011255 28665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:06:20.012284 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 28665
I20260806 21:06:20.012440 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:06:20.012804 28551 external_mini_cluster.cc:1428] Setting key 50833b84ed61d9266ebce3073915fe13
Aug 06 21:06:20 dist-test-slave-69rg krb5kdc[28558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050379, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.225.194@KRBTEST.COM; defaulting to no policy
I20260806 21:06:20.032130 28642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:20.014773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:56403 (local address 127.27.225.254:42211)
0806 21:06:20.015796 (+ 1023us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:20.015803 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:20.016526 (+ 723us) server_negotiation.cc:407] Connection header received
0806 21:06:20.017473 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:20.017483 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:20.017545 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:20.017687 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:20.019221 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:20.019827 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:20.021424 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:20.021775 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:20.022059 (+ 284us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:20.025992 (+ 3933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:20.025999 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:20.026006 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:20.026055 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:20.028377 (+ 2322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:20.029424 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:20.029432 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:20.029437 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:20.029548 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:20.030209 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:20.030212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:20.030213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:20.030676 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:20.030797 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:20.031493 (+ 696us) server_negotiation.cc:299] Negotiation successful
0806 21:06:20.031955 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265}
I20260806 21:06:20.033475 28797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:20.015390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:42211 (local address 127.27.225.193:56403)
0806 21:06:20.016250 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:20.016313 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:20.017170 (+ 857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:20.017963 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:20.017971 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:20.018405 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:20.018948 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:20.018964 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:20.019993 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:20.019999 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:20.021178 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:20.021192 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:20.022133 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:20.022140 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:20.022419 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:20.023480 (+ 1061us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:20.023511 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:20.025566 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:20.028816 (+ 3250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:20.028824 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:20.028841 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:20.029273 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:20.029724 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:20.029727 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:20.029729 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:20.030030 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:20.030841 (+ 811us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:20.030849 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:20.031155 (+ 306us) client_negotiation.cc:769] Sending connection context
0806 21:06:20.031458 (+ 303us) client_negotiation.cc:240] Negotiation successful
0806 21:06:20.032126 (+ 668us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":192,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.194@KRBTEST.COM" created.
I20260806 21:06:20.035737 28795 heartbeater.cc:347] Connected to a master server at 127.27.225.254:42211
I20260806 21:06:20.036070 28795 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:20.036753 28795 heartbeater.cc:511] Master 127.27.225.254:42211 requested a full tablet report, sending...
I20260806 21:06:20.039848 28603 ts_manager.cc:194] Registered new tserver with Master: 2355df5ef8784148b15df257c39ba222 (127.27.225.193:39955)
I20260806 21:06:20.042207 28603 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.193@KRBTEST.COM'} at 127.27.225.193:56403
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
I20260806 21:06:20.057914 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:42211
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:36021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:20.183861 28800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:20.184274 28800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:20.184356 28800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:20.189138 28800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:20.189221 28800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:20.189288 28800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:06:20.195557 28800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:36021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:42211
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:20.196707 28800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:20.197764 28800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:20.208148 28805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:20.208240 28806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:20.208328 28800 server_base.cc:1034] running on GCE node
W20260806 21:06:20.208547 28808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:20.208985 28800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:20.209700 28800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:20.210922 28800 hybrid_clock.cc:648] HybridClock initialized: now 1786050380210878 us; error 58 us; skew 500 ppm
Aug 06 21:06:20 dist-test-slave-69rg krb5kdc[28558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050380, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:20.214042 28800 init.cc:377] Logged in from keytab as kudu/127.27.225.194@KRBTEST.COM (short username kudu)
I20260806 21:06:20.214943 28800 webserver.cc:460] Webserver started at http://127.27.225.194:36369/ using document root <none> and password file <none>
I20260806 21:06:20.215500 28800 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:20.215562 28800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:20.215770 28800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:20.218142 28800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "69eb2168ea7a4d348b8a8807127d5e0b"
format_stamp: "Formatted at 2026-08-06 21:06:20 on dist-test-slave-69rg"
server_key: "df58f7272aaf0c6db90c2f04ffa23f1a"
server_key_iv: "51eedbeea81f37273bb426e3e6a9c699"
server_key_version: "encryptionkey@0"
I20260806 21:06:20.218803 28800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "69eb2168ea7a4d348b8a8807127d5e0b"
format_stamp: "Formatted at 2026-08-06 21:06:20 on dist-test-slave-69rg"
server_key: "df58f7272aaf0c6db90c2f04ffa23f1a"
server_key_iv: "51eedbeea81f37273bb426e3e6a9c699"
server_key_version: "encryptionkey@0"
I20260806 21:06:20.223145 28800 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260806 21:06:20.225783 28815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:20.227257 28800 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 21:06:20.227407 28800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "69eb2168ea7a4d348b8a8807127d5e0b"
format_stamp: "Formatted at 2026-08-06 21:06:20 on dist-test-slave-69rg"
server_key: "df58f7272aaf0c6db90c2f04ffa23f1a"
server_key_iv: "51eedbeea81f37273bb426e3e6a9c699"
server_key_version: "encryptionkey@0"
I20260806 21:06:20.227540 28800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:20.242864 28800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:20.246698 28800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:20.246910 28800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:20.247545 28800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:20.248407 28800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:20.248512 28800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:20.248571 28800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:20.248643 28800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 21:06:20.258642 28800 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:39331
I20260806 21:06:20.258668 28930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:39331 every 8 connection(s)
I20260806 21:06:20.259618 28800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:06:20.266878 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 28800
I20260806 21:06:20.266978 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:06:20.267285 28551 external_mini_cluster.cc:1428] Setting key f572dd0d008526479326052ed5881530
Aug 06 21:06:20 dist-test-slave-69rg krb5kdc[28558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050380, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:20.275503 28642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:20.261736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:33171 (local address 127.27.225.254:42211)
0806 21:06:20.261993 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:20.261997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:20.262888 (+ 891us) server_negotiation.cc:407] Connection header received
0806 21:06:20.264713 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:20.264717 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:20.264795 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:20.264899 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:20.266239 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:20.266802 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:20.268207 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:20.268447 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:20.268564 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:20.271460 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:20.271466 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:20.271469 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:20.271509 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:20.273076 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:20.273768 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:20.273772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:20.273774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:20.273923 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:20.274299 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:20.274303 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:20.274304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:20.274524 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:20.274633 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:20.275175 (+ 542us) server_negotiation.cc:299] Negotiation successful
0806 21:06:20.275322 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260806 21:06:20.276372 28933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:20.262160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:42211 (local address 127.27.225.194:33171)
0806 21:06:20.262694 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:20.262741 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:20.264455 (+ 1714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:20.265100 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:20.265108 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:20.265533 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:20.266071 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:20.266086 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:20.266946 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:20.266952 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:20.268079 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:20.268090 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:20.268612 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:20.268616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:20.268863 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:20.269597 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:20.269615 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:20.271212 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:20.273288 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:20.273294 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:20.273313 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:20.273607 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:20.274029 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:20.274032 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:20.274034 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:20.274194 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:20.274631 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:20.274639 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:20.274919 (+ 280us) client_negotiation.cc:769] Sending connection context
0806 21:06:20.275108 (+ 189us) client_negotiation.cc:240] Negotiation successful
0806 21:06:20.275466 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":143,"threads_started":1}
WARNING: no policy specified for kudu/127.27.225.195@KRBTEST.COM; defaulting to no policy
I20260806 21:06:20.277881 28931 heartbeater.cc:347] Connected to a master server at 127.27.225.254:42211
I20260806 21:06:20.278231 28931 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:20.278883 28931 heartbeater.cc:511] Master 127.27.225.254:42211 requested a full tablet report, sending...
I20260806 21:06:20.280313 28603 ts_manager.cc:194] Registered new tserver with Master: 69eb2168ea7a4d348b8a8807127d5e0b (127.27.225.194:39331)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.195@KRBTEST.COM" created.
I20260806 21:06:20.281055 28603 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.194@KRBTEST.COM'} at 127.27.225.194:33171
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
I20260806 21:06:20.302371 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:42211
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:36021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:20.427345 28936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:20.427706 28936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:20.427757 28936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:20.433144 28936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:20.433248 28936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:20.433318 28936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:06:20.439393 28936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:36021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:42211
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:20.440650 28936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:20.441802 28936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:20.452636 28942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:20.452623 28945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:20.452632 28941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:20.453131 28936 server_base.cc:1034] running on GCE node
I20260806 21:06:20.453562 28936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:20.454372 28936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:20.455565 28936 hybrid_clock.cc:648] HybridClock initialized: now 1786050380455543 us; error 38 us; skew 500 ppm
Aug 06 21:06:20 dist-test-slave-69rg krb5kdc[28558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050380, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:20.459045 28936 init.cc:377] Logged in from keytab as kudu/127.27.225.195@KRBTEST.COM (short username kudu)
I20260806 21:06:20.460214 28936 webserver.cc:460] Webserver started at http://127.27.225.195:42275/ using document root <none> and password file <none>
I20260806 21:06:20.460846 28936 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:20.460942 28936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:20.461194 28936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:20.463517 28936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "aac47a71a366497fb33e8dc436514fa7"
format_stamp: "Formatted at 2026-08-06 21:06:20 on dist-test-slave-69rg"
server_key: "c552cb0238bea4927d9cbbe798009ceb"
server_key_iv: "142eed71e9cc7d97a59eb52af570518f"
server_key_version: "encryptionkey@0"
I20260806 21:06:20.464193 28936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "aac47a71a366497fb33e8dc436514fa7"
format_stamp: "Formatted at 2026-08-06 21:06:20 on dist-test-slave-69rg"
server_key: "c552cb0238bea4927d9cbbe798009ceb"
server_key_iv: "142eed71e9cc7d97a59eb52af570518f"
server_key_version: "encryptionkey@0"
I20260806 21:06:20.468931 28936 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 21:06:20.471836 28952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:20.473181 28936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:06:20.473353 28936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "aac47a71a366497fb33e8dc436514fa7"
format_stamp: "Formatted at 2026-08-06 21:06:20 on dist-test-slave-69rg"
server_key: "c552cb0238bea4927d9cbbe798009ceb"
server_key_iv: "142eed71e9cc7d97a59eb52af570518f"
server_key_version: "encryptionkey@0"
I20260806 21:06:20.473490 28936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:20.519883 28936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:20.524010 28936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:20.524281 28936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:20.525004 28936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:20.526209 28936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:20.526299 28936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:20.526376 28936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:20.526417 28936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:20.537127 28936 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:39247
I20260806 21:06:20.537150 29065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:39247 every 8 connection(s)
I20260806 21:06:20.538167 28936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:06:20.541899 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 28936
I20260806 21:06:20.542024 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:06:20.542313 28551 external_mini_cluster.cc:1428] Setting key ef78e12812948eb857b691cdb22ab6c1
Aug 06 21:06:20 dist-test-slave-69rg krb5kdc[28558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050380, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:20.554142 28642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:20.540317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:41555 (local address 127.27.225.254:42211)
0806 21:06:20.540479 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:20.540483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:20.541435 (+ 952us) server_negotiation.cc:407] Connection header received
0806 21:06:20.542961 (+ 1526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:20.542965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:20.543025 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:20.543096 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:20.544553 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:20.545375 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:20.546909 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:20.547157 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:20.547256 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:20.550101 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:20.550105 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:20.550108 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:20.550136 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:20.551661 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:20.552255 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:20.552258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:20.552260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:20.552341 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:20.552789 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:20.552792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:20.552793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:20.553081 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:20.553177 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:20.553793 (+ 616us) server_negotiation.cc:299] Negotiation successful
0806 21:06:20.553996 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 21:06:20.554979 29068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:20.540714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:42211 (local address 127.27.225.195:41555)
0806 21:06:20.541216 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:20.541308 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:20.542717 (+ 1409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:20.543331 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:20.543342 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:20.543776 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:20.544396 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:20.544411 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:20.545540 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:20.545545 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:20.546725 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:20.546737 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:20.547337 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:20.547340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:20.547527 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:20.548164 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:20.548181 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:20.549915 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:20.551789 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:20.551795 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:20.551809 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:20.552107 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:20.552452 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:20.552454 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:20.552456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:20.552623 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:20.553214 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:20.553220 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:20.553506 (+ 286us) client_negotiation.cc:769] Sending connection context
0806 21:06:20.553702 (+ 196us) client_negotiation.cc:240] Negotiation successful
0806 21:06:20.554053 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":134,"threads_started":1}
I20260806 21:06:20.556653 29066 heartbeater.cc:347] Connected to a master server at 127.27.225.254:42211
I20260806 21:06:20.556959 29066 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:20.557611 29066 heartbeater.cc:511] Master 127.27.225.254:42211 requested a full tablet report, sending...
I20260806 21:06:20.558717 28603 ts_manager.cc:194] Registered new tserver with Master: aac47a71a366497fb33e8dc436514fa7 (127.27.225.195:39247)
I20260806 21:06:20.559309 28603 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.195@KRBTEST.COM'} at 127.27.225.195:41555
I20260806 21:06:20.570288 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 21:06:20 dist-test-slave-69rg krb5kdc[28558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050380, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 21:06:20 dist-test-slave-69rg krb5kdc[28558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050380, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.225.193@KRBTEST.COM
I20260806 21:06:20.602938 29073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:20.591186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45858 (local address 127.27.225.193:39955)
0806 21:06:20.591495 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:20.591502 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:20.591529 (+ 27us) server_negotiation.cc:407] Connection header received
0806 21:06:20.591658 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:20.591664 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:20.591864 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:20.591988 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:20.592769 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:20.593560 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:20.594592 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:20.595000 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:20.595102 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:20.598325 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:20.598341 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:20.598358 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:20.598394 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:20.600416 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:20.601231 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:20.601239 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:20.601240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:20.601360 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:20.601752 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:20.601756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:20.601757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:20.602153 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:20.602350 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:20.602606 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 21:06:20.602783 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":81,"threads_started":1}
W20260806 21:06:20.604043 28708 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45858
Aug 06 21:06:20 dist-test-slave-69rg krb5kdc[28558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 21:06:20 dist-test-slave-69rg krb5kdc[28558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.193@KRBTEST.COM
I20260806 21:06:20.636989 29073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:20.626385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45870 (local address 127.27.225.193:39955)
0806 21:06:20.626590 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:20.626594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:20.626686 (+ 92us) server_negotiation.cc:407] Connection header received
0806 21:06:20.626837 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:20.626840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:20.626894 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:20.626964 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:20.627921 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:20.628462 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:20.629407 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:20.629655 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:20.629760 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:20.633252 (+ 3492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:20.633256 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:20.633258 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:20.633293 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:20.634949 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:20.635495 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:20.635499 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:20.635500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:20.635613 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:20.635946 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:20.635950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:20.635951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:20.636198 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:20.636334 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:20.636608 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 21:06:20.636780 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 21:06:20.638458 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 28665
I20260806 21:06:20.646795 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 28800
I20260806 21:06:20.653508 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 28936
I20260806 21:06:20.660063 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 28571
2026-08-06T21:06:20Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2536 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:06:20 dist-test-slave-69rg krb5kdc[29080](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:06:20 dist-test-slave-69rg krb5kdc[29080](info): set up 2 sockets
Aug 06 21:06:20 dist-test-slave-69rg krb5kdc[29080](info): commencing operation
krb5kdc: starting...
W20260806 21:06:21.746731 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:06:21 dist-test-slave-69rg krb5kdc[29080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:06:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:06:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:06:21.890501 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:36279
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:45889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:36279
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:22.012418 29095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:22.012789 29095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:22.012872 29095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:22.018296 29095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:06:22.018788 29095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:22.018860 29095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:22.018918 29095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:06:22.018955 29095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:06:22.026151 29095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:45889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:36279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:36279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:22.027498 29095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:22.029070 29095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:22.039175 29103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:22.039408 29095 server_base.cc:1034] running on GCE node
W20260806 21:06:22.039177 29101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:22.039232 29100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:22.040776 29095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:22.042321 29095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:22.043895 29095 hybrid_clock.cc:648] HybridClock initialized: now 1786050382043656 us; error 74 us; skew 500 ppm
Aug 06 21:06:22 dist-test-slave-69rg krb5kdc[29080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050382, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:22.047330 29095 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:06:22.048429 29095 webserver.cc:460] Webserver started at http://127.27.225.254:38441/ using document root <none> and password file <none>
I20260806 21:06:22.049057 29095 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:22.049163 29095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:22.049422 29095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:22.051810 29095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "7c80bde07e9e4462844e815759372a2f"
format_stamp: "Formatted at 2026-08-06 21:06:22 on dist-test-slave-69rg"
server_key: "bd0110ea8252593b6d58f6f0be7a105e"
server_key_iv: "53c04f2adaed78fb98972a98c6ee6a54"
server_key_version: "encryptionkey@0"
I20260806 21:06:22.052515 29095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "7c80bde07e9e4462844e815759372a2f"
format_stamp: "Formatted at 2026-08-06 21:06:22 on dist-test-slave-69rg"
server_key: "bd0110ea8252593b6d58f6f0be7a105e"
server_key_iv: "53c04f2adaed78fb98972a98c6ee6a54"
server_key_version: "encryptionkey@0"
I20260806 21:06:22.057737 29095 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260806 21:06:22.061383 29110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:22.063356 29095 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 21:06:22.063613 29095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "7c80bde07e9e4462844e815759372a2f"
format_stamp: "Formatted at 2026-08-06 21:06:22 on dist-test-slave-69rg"
server_key: "bd0110ea8252593b6d58f6f0be7a105e"
server_key_iv: "53c04f2adaed78fb98972a98c6ee6a54"
server_key_version: "encryptionkey@0"
I20260806 21:06:22.063772 29095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:22.095724 29095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:22.099793 29095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:22.100097 29095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:22.113596 29095 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:36279
I20260806 21:06:22.113992 29162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:36279 every 8 connection(s)
I20260806 21:06:22.115741 29095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:06:22.119899 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:22.120405 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 29095
I20260806 21:06:22.120532 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:06:22.120844 28551 external_mini_cluster.cc:1428] Setting key 972b3ac0a87873114772dcda94503a74
I20260806 21:06:22.130846 29163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c80bde07e9e4462844e815759372a2f: Bootstrap starting.
I20260806 21:06:22.135202 29163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c80bde07e9e4462844e815759372a2f: Neither blocks nor log segments found. Creating new log.
Aug 06 21:06:22 dist-test-slave-69rg krb5kdc[29080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:22.136605 29163 log.cc:824] T 00000000000000000000000000000000 P 7c80bde07e9e4462844e815759372a2f: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:22.139424 29163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c80bde07e9e4462844e815759372a2f: No bootstrap required, opened a new log
I20260806 21:06:22.144779 29166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:22.122469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41194 (local address 127.27.225.254:36279)
0806 21:06:22.122972 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:22.122990 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:22.123028 (+ 38us) server_negotiation.cc:407] Connection header received
0806 21:06:22.123930 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:22.123955 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:22.124360 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:22.124956 (+ 596us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:22.126128 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:22.129971 (+ 3843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:22.131870 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:22.132771 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:22.132952 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:22.137222 (+ 4270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:22.137229 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:22.137246 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:22.137298 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:22.140260 (+ 2962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:22.140925 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:22.140932 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:22.140937 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:22.141083 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:22.141590 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:22.141594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:22.141595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:22.142389 (+ 794us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:22.142589 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:22.143243 (+ 654us) server_negotiation.cc:299] Negotiation successful
0806 21:06:22.143768 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":191,"threads_started":1}
I20260806 21:06:22.145172 29163 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c80bde07e9e4462844e815759372a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c80bde07e9e4462844e815759372a2f" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 36279 } }
I20260806 21:06:22.145524 29163 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c80bde07e9e4462844e815759372a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:22.145591 29163 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c80bde07e9e4462844e815759372a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c80bde07e9e4462844e815759372a2f, State: Initialized, Role: FOLLOWER
I20260806 21:06:22.146216 29163 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c80bde07e9e4462844e815759372a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c80bde07e9e4462844e815759372a2f" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 36279 } }
I20260806 21:06:22.146634 29163 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c80bde07e9e4462844e815759372a2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:06:22.146739 29163 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c80bde07e9e4462844e815759372a2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:06:22.146867 29163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c80bde07e9e4462844e815759372a2f [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:22.148053 29163 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c80bde07e9e4462844e815759372a2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c80bde07e9e4462844e815759372a2f" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 36279 } }
I20260806 21:06:22.148631 29163 leader_election.cc:302] T 00000000000000000000000000000000 P 7c80bde07e9e4462844e815759372a2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c80bde07e9e4462844e815759372a2f; no voters:
I20260806 21:06:22.149026 29163 leader_election.cc:288] T 00000000000000000000000000000000 P 7c80bde07e9e4462844e815759372a2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:06:22.149209 29168 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c80bde07e9e4462844e815759372a2f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:22.149483 29168 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c80bde07e9e4462844e815759372a2f [term 1 LEADER]: Becoming Leader. State: Replica: 7c80bde07e9e4462844e815759372a2f, State: Running, Role: LEADER
I20260806 21:06:22.149911 29168 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c80bde07e9e4462844e815759372a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c80bde07e9e4462844e815759372a2f" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 36279 } }
I20260806 21:06:22.150833 29163 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c80bde07e9e4462844e815759372a2f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:06:22.151742 29169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c80bde07e9e4462844e815759372a2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c80bde07e9e4462844e815759372a2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c80bde07e9e4462844e815759372a2f" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 36279 } } }
I20260806 21:06:22.151911 29169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c80bde07e9e4462844e815759372a2f [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:22.152189 29170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c80bde07e9e4462844e815759372a2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c80bde07e9e4462844e815759372a2f. Latest consensus state: current_term: 1 leader_uuid: "7c80bde07e9e4462844e815759372a2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c80bde07e9e4462844e815759372a2f" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 36279 } } }
I20260806 21:06:22.152292 29170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c80bde07e9e4462844e815759372a2f [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:22.152393 29177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:06:22.156602 29177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:06:22.164757 29177 catalog_manager.cc:1347] Generated new cluster ID: 1f88327ea82a481498d14b6fe1c80295
I20260806 21:06:22.164882 29177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:06:22.195226 29177 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:06:22.196588 29177 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:06:22.204941 29177 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c80bde07e9e4462844e815759372a2f: Generated new TSK 0
I20260806 21:06:22.206017 29177 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.225.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
I20260806 21:06:22.257311 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:36279
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:45889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:22.387708 29190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:22.388006 29190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:22.388055 29190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:22.392288 29190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:22.392372 29190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:22.392472 29190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:06:22.398517 29190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:45889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:36279
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:22.399739 29190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:22.400833 29190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:22.411798 29196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:22.411798 29198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:22.411823 29195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:22.412087 29190 server_base.cc:1034] running on GCE node
I20260806 21:06:22.412859 29190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:22.413538 29190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:22.414736 29190 hybrid_clock.cc:648] HybridClock initialized: now 1786050382414699 us; error 51 us; skew 500 ppm
Aug 06 21:06:22 dist-test-slave-69rg krb5kdc[29080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050382, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:22.417773 29190 init.cc:377] Logged in from keytab as kudu/127.27.225.193@KRBTEST.COM (short username kudu)
I20260806 21:06:22.418818 29190 webserver.cc:460] Webserver started at http://127.27.225.193:39825/ using document root <none> and password file <none>
I20260806 21:06:22.419386 29190 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:22.419451 29190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:22.419631 29190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:22.421991 29190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "ffa112f475c542d0a0df31a04ba9cd33"
format_stamp: "Formatted at 2026-08-06 21:06:22 on dist-test-slave-69rg"
server_key: "c523e1f749e919aff50180407e498fff"
server_key_iv: "a80e5fdecfe401d5d36d7f76f03a5ce1"
server_key_version: "encryptionkey@0"
I20260806 21:06:22.422859 29190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "ffa112f475c542d0a0df31a04ba9cd33"
format_stamp: "Formatted at 2026-08-06 21:06:22 on dist-test-slave-69rg"
server_key: "c523e1f749e919aff50180407e498fff"
server_key_iv: "a80e5fdecfe401d5d36d7f76f03a5ce1"
server_key_version: "encryptionkey@0"
I20260806 21:06:22.428012 29190 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 21:06:22.431185 29205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:22.432654 29190 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 21:06:22.432873 29190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "ffa112f475c542d0a0df31a04ba9cd33"
format_stamp: "Formatted at 2026-08-06 21:06:22 on dist-test-slave-69rg"
server_key: "c523e1f749e919aff50180407e498fff"
server_key_iv: "a80e5fdecfe401d5d36d7f76f03a5ce1"
server_key_version: "encryptionkey@0"
I20260806 21:06:22.433023 29190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:22.457641 29190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:22.461575 29190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:22.461863 29190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:22.462566 29190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:22.463551 29190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:22.463622 29190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:22.463685 29190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:22.463724 29190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:22.475481 29190 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:36509
I20260806 21:06:22.475502 29319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:36509 every 8 connection(s)
I20260806 21:06:22.476611 29190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:06:22.486261 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 29190
I20260806 21:06:22.486408 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:06:22.486840 28551 external_mini_cluster.cc:1428] Setting key ef09cbdd63c33385df2baa6a5463a5d5
Aug 06 21:06:22 dist-test-slave-69rg krb5kdc[29080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050382, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:22.494732 29166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:22.479021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:58977 (local address 127.27.225.254:36279)
0806 21:06:22.479216 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:22.479222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:22.480412 (+ 1190us) server_negotiation.cc:407] Connection header received
0806 21:06:22.481946 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:22.481949 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:22.482006 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:22.482137 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:22.483547 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:22.484173 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:22.485775 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:22.486245 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:22.486435 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:22.489730 (+ 3295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:22.489737 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:22.489744 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:22.489784 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:22.491692 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:22.492527 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:22.492530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:22.492532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:22.492630 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:22.493170 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:22.493174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:22.493175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:22.493407 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:22.493514 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:22.494349 (+ 835us) server_negotiation.cc:299] Negotiation successful
0806 21:06:22.494552 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 21:06:22.496068 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:22.479577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:36279 (local address 127.27.225.193:58977)
0806 21:06:22.480185 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:22.480235 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:22.481511 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:22.482368 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:22.482376 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:22.482812 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:22.483345 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:22.483360 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:22.484393 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:22.484396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:22.485528 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:22.485542 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:22.486558 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:22.486562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:22.486745 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:22.487722 (+ 977us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:22.487760 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:22.489513 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:22.491862 (+ 2349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:22.491871 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:22.491895 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:22.492360 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:22.492767 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:22.492773 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:22.492778 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:22.493005 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:22.493560 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:22.493570 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:22.494005 (+ 435us) client_negotiation.cc:769] Sending connection context
0806 21:06:22.494282 (+ 277us) client_negotiation.cc:240] Negotiation successful
0806 21:06:22.494733 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":159,"threads_started":1}
I20260806 21:06:22.497635 29321 heartbeater.cc:347] Connected to a master server at 127.27.225.254:36279
I20260806 21:06:22.497979 29321 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:22.498555 29321 heartbeater.cc:511] Master 127.27.225.254:36279 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.27.225.194@KRBTEST.COM; defaulting to no policy
I20260806 21:06:22.500465 29127 ts_manager.cc:194] Registered new tserver with Master: ffa112f475c542d0a0df31a04ba9cd33 (127.27.225.193:36509)
I20260806 21:06:22.501646 29127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.193@KRBTEST.COM'} at 127.27.225.193:58977
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
I20260806 21:06:22.527519 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:36279
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:45889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:22.656883 29326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:22.657214 29326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:22.657305 29326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:22.661590 29326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:22.661717 29326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:22.661861 29326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:06:22.668210 29326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:45889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:36279
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:22.669523 29326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:22.670802 29326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:22.681576 29334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:22.681579 29332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:22.681576 29331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:22.681661 29326 server_base.cc:1034] running on GCE node
I20260806 21:06:22.682521 29326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:22.683346 29326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:22.684536 29326 hybrid_clock.cc:648] HybridClock initialized: now 1786050382684484 us; error 68 us; skew 500 ppm
Aug 06 21:06:22 dist-test-slave-69rg krb5kdc[29080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050382, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:22.687879 29326 init.cc:377] Logged in from keytab as kudu/127.27.225.194@KRBTEST.COM (short username kudu)
I20260806 21:06:22.688854 29326 webserver.cc:460] Webserver started at http://127.27.225.194:40805/ using document root <none> and password file <none>
I20260806 21:06:22.689420 29326 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:22.689504 29326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:22.689693 29326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:22.691900 29326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "84e782d1156b4aa8a88391410a8ca17a"
format_stamp: "Formatted at 2026-08-06 21:06:22 on dist-test-slave-69rg"
server_key: "0423ec939e06144108f6e0af3ae7bff5"
server_key_iv: "14060587592a98c2cac3f8d002bd7132"
server_key_version: "encryptionkey@0"
I20260806 21:06:22.692448 29326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "84e782d1156b4aa8a88391410a8ca17a"
format_stamp: "Formatted at 2026-08-06 21:06:22 on dist-test-slave-69rg"
server_key: "0423ec939e06144108f6e0af3ae7bff5"
server_key_iv: "14060587592a98c2cac3f8d002bd7132"
server_key_version: "encryptionkey@0"
I20260806 21:06:22.696892 29326 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 21:06:22.699839 29341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:22.701448 29326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:06:22.701578 29326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "84e782d1156b4aa8a88391410a8ca17a"
format_stamp: "Formatted at 2026-08-06 21:06:22 on dist-test-slave-69rg"
server_key: "0423ec939e06144108f6e0af3ae7bff5"
server_key_iv: "14060587592a98c2cac3f8d002bd7132"
server_key_version: "encryptionkey@0"
I20260806 21:06:22.701748 29326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:22.717484 29326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:22.721060 29326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:22.721331 29326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:22.722059 29326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:22.722926 29326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:22.722973 29326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:22.723042 29326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:22.723080 29326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:22.732476 29326 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:44153
I20260806 21:06:22.732496 29456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:44153 every 8 connection(s)
I20260806 21:06:22.733450 29326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:06:22.736768 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 29326
I20260806 21:06:22.736882 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:06:22.737218 28551 external_mini_cluster.cc:1428] Setting key 2e09c6b9b42c3e6b22dcca8510cd95df
Aug 06 21:06:22 dist-test-slave-69rg krb5kdc[29080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050382, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.225.195@KRBTEST.COM; defaulting to no policy
I20260806 21:06:22.748579 29166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:22.735422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:38817 (local address 127.27.225.254:36279)
0806 21:06:22.735553 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:22.735557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:22.736363 (+ 806us) server_negotiation.cc:407] Connection header received
0806 21:06:22.737684 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:22.737689 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:22.737755 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:22.737873 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:22.739336 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:22.739861 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:22.741396 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:22.741783 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:22.741931 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:22.744777 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:22.744780 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:22.744782 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:22.744809 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:22.746348 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:22.746924 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:22.746927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:22.746928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:22.747009 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:22.747370 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:22.747375 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:22.747376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:22.747576 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:22.747686 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:22.748269 (+ 583us) server_negotiation.cc:299] Negotiation successful
0806 21:06:22.748403 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 21:06:22.749492 29459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:22.735754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:36279 (local address 127.27.225.194:38817)
0806 21:06:22.736209 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:22.736250 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:22.737395 (+ 1145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:22.738055 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:22.738075 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:22.738521 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:22.739173 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:22.739195 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:22.739986 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:22.739992 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:22.741272 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:22.741282 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:22.741955 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:22.741958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:22.742105 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:22.742750 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:22.742776 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:22.744560 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:22.746478 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:22.746485 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:22.746503 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:22.746802 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:22.747106 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:22.747109 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:22.747111 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:22.747275 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:22.747691 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:22.747700 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:22.748024 (+ 324us) client_negotiation.cc:769] Sending connection context
0806 21:06:22.748223 (+ 199us) client_negotiation.cc:240] Negotiation successful
0806 21:06:22.748584 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":117,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.195@KRBTEST.COM" created.
I20260806 21:06:22.750942 29457 heartbeater.cc:347] Connected to a master server at 127.27.225.254:36279
I20260806 21:06:22.751223 29457 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:22.751881 29457 heartbeater.cc:511] Master 127.27.225.254:36279 requested a full tablet report, sending...
I20260806 21:06:22.753053 29127 ts_manager.cc:194] Registered new tserver with Master: 84e782d1156b4aa8a88391410a8ca17a (127.27.225.194:44153)
I20260806 21:06:22.754024 29127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.194@KRBTEST.COM'} at 127.27.225.194:38817
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
I20260806 21:06:22.770186 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:36279
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:45889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:22.888156 29462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:22.888474 29462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:22.888523 29462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:22.892833 29462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:22.892910 29462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:22.892974 29462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:06:22.898655 29462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:45889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:36279
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:22.899714 29462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:22.900692 29462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:22.910768 29468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:22.910900 29462 server_base.cc:1034] running on GCE node
W20260806 21:06:22.910727 29470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:22.910770 29467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:22.911517 29462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:22.912285 29462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:22.913501 29462 hybrid_clock.cc:648] HybridClock initialized: now 1786050382913482 us; error 32 us; skew 500 ppm
Aug 06 21:06:22 dist-test-slave-69rg krb5kdc[29080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050382, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:22.916864 29462 init.cc:377] Logged in from keytab as kudu/127.27.225.195@KRBTEST.COM (short username kudu)
I20260806 21:06:22.918059 29462 webserver.cc:460] Webserver started at http://127.27.225.195:35983/ using document root <none> and password file <none>
I20260806 21:06:22.918730 29462 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:22.918826 29462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:22.919077 29462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:22.921489 29462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "b8e88404f0b04b5a9efe2875af0204e5"
format_stamp: "Formatted at 2026-08-06 21:06:22 on dist-test-slave-69rg"
server_key: "03bfaa95a3c4dacb8152e4642be68863"
server_key_iv: "dc81f409bd081ad969136bdf04fbd5cb"
server_key_version: "encryptionkey@0"
I20260806 21:06:22.922274 29462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "b8e88404f0b04b5a9efe2875af0204e5"
format_stamp: "Formatted at 2026-08-06 21:06:22 on dist-test-slave-69rg"
server_key: "03bfaa95a3c4dacb8152e4642be68863"
server_key_iv: "dc81f409bd081ad969136bdf04fbd5cb"
server_key_version: "encryptionkey@0"
I20260806 21:06:22.926506 29462 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 21:06:22.929463 29477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:22.930923 29462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:06:22.931074 29462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "b8e88404f0b04b5a9efe2875af0204e5"
format_stamp: "Formatted at 2026-08-06 21:06:22 on dist-test-slave-69rg"
server_key: "03bfaa95a3c4dacb8152e4642be68863"
server_key_iv: "dc81f409bd081ad969136bdf04fbd5cb"
server_key_version: "encryptionkey@0"
I20260806 21:06:22.931202 29462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:22.943459 29462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:22.947570 29462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:22.947834 29462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:22.948509 29462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:22.949433 29462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:22.949505 29462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:22.949576 29462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:22.949616 29462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:22.960711 29462 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:34321
I20260806 21:06:22.960783 29590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:34321 every 8 connection(s)
I20260806 21:06:22.962224 29462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:06:22.968621 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 29462
I20260806 21:06:22.968823 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:06:22.969147 28551 external_mini_cluster.cc:1428] Setting key 299580bf89eef0e1ab78ce4e01cca249
Aug 06 21:06:22 dist-test-slave-69rg krb5kdc[29080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050382, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:22.979657 29166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:22.964809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:56477 (local address 127.27.225.254:36279)
0806 21:06:22.964977 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:22.964982 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:22.966090 (+ 1108us) server_negotiation.cc:407] Connection header received
0806 21:06:22.967164 (+ 1074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:22.967168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:22.967238 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:22.967352 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:22.968770 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:22.969683 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:22.971404 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:22.971764 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:22.971854 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:22.975186 (+ 3332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:22.975190 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:22.975193 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:22.975259 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:22.977204 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:22.978000 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:22.978015 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:22.978017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:22.978152 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:22.978526 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:22.978528 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:22.978529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:22.978746 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:22.978849 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:22.979403 (+ 554us) server_negotiation.cc:299] Negotiation successful
0806 21:06:22.979524 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 21:06:22.980574 29593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:22.965225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:36279 (local address 127.27.225.195:56477)
0806 21:06:22.965911 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:22.965963 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:22.966918 (+ 955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:22.967570 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:22.967577 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:22.968008 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:22.968578 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:22.968595 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:22.969902 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:22.969912 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:22.971270 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:22.971284 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:22.971922 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:22.971925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:22.972097 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:22.972783 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:22.972810 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:22.974883 (+ 2073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:22.977366 (+ 2483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:22.977374 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:22.977389 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:22.977857 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:22.978258 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:22.978260 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:22.978261 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:22.978432 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:22.978837 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:22.978843 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:22.979116 (+ 273us) client_negotiation.cc:769] Sending connection context
0806 21:06:22.979293 (+ 177us) client_negotiation.cc:240] Negotiation successful
0806 21:06:22.979585 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":155,"threads_started":1}
I20260806 21:06:22.982018 29591 heartbeater.cc:347] Connected to a master server at 127.27.225.254:36279
I20260806 21:06:22.982378 29591 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:22.983306 29591 heartbeater.cc:511] Master 127.27.225.254:36279 requested a full tablet report, sending...
I20260806 21:06:22.984596 29127 ts_manager.cc:194] Registered new tserver with Master: b8e88404f0b04b5a9efe2875af0204e5 (127.27.225.195:34321)
I20260806 21:06:22.985183 29127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.195@KRBTEST.COM'} at 127.27.225.195:56477
I20260806 21:06:22.996624 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:06:23.012197 29166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:23.001268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41198 (local address 127.27.225.254:36279)
0806 21:06:23.001455 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:23.001461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:23.001630 (+ 169us) server_negotiation.cc:407] Connection header received
0806 21:06:23.001751 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:23.001753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:23.001804 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:23.001939 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:23.002682 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.003267 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.004165 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.004411 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.004684 (+ 273us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:23.006564 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:23.006568 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:23.006570 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:23.006601 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:23.008945 (+ 2344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:23.010328 (+ 1383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:23.010336 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:23.010338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:23.010477 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:23.010922 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:23.010926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:23.010929 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:23.011229 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:23.011549 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:23.011858 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 21:06:23.012012 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 21:06:23.020043 29127 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41198:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:06:23.023725 29127 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:06:23.041519 29607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:23.035735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60536 (local address 127.27.225.193:36509)
0806 21:06:23.036634 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:23.036642 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:23.036679 (+ 37us) server_negotiation.cc:407] Connection header received
0806 21:06:23.036753 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:23.036758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:23.037065 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:23.037267 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:23.037492 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.038401 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.040244 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.040782 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.040922 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:23.041135 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:23.041247 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 21:06:23.041349 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":797,"thread_start_us":73,"threads_started":1}
I20260806 21:06:23.041512 29606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:23.035065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:36509 (local address 127.0.0.1:60536)
0806 21:06:23.035952 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:23.036008 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:23.036152 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:23.037257 (+ 1105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:23.037260 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:23.037279 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:23.037375 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:23.037382 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.038540 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.038547 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:23.040046 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:23.040057 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.040912 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.040916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:23.041048 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:23.041089 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 21:06:23.041217 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 21:06:23.041353 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"mutex_wait_us":50,"spinlock_wait_cycles":384,"thread_start_us":117,"threads_started":1}
I20260806 21:06:23.041922 29608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:23.035911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48044 (local address 127.27.225.195:34321)
0806 21:06:23.036474 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:23.036485 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:23.036505 (+ 20us) server_negotiation.cc:407] Connection header received
0806 21:06:23.036571 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:23.036575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:23.036765 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:23.036903 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:23.037281 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.038541 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.040186 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.041034 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.041147 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:23.041257 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:23.041619 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 21:06:23.041755 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":119,"threads_started":1}
I20260806 21:06:23.042122 29604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:23.035072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:44153 (local address 127.0.0.1:58946)
0806 21:06:23.036029 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:23.036043 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:23.036152 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:23.036969 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:23.036974 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:23.036998 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:23.037144 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:23.037151 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.038732 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.038734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:23.039842 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:23.039852 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.041213 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.041217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:23.041314 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:23.041338 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 21:06:23.041408 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 21:06:23.041492 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"mutex_wait_us":219,"spinlock_wait_cycles":9984,"thread_start_us":92,"threads_started":1}
I20260806 21:06:23.042253 29602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:23.035389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:34321 (local address 127.0.0.1:48044)
0806 21:06:23.036235 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:23.036251 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:23.036346 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:23.037047 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:23.037050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:23.037063 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:23.037144 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:23.037151 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.038701 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.038704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:23.040070 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:23.040076 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.041310 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.041313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:23.041410 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:23.041429 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 21:06:23.041485 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 21:06:23.041545 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"mutex_wait_us":68,"thread_start_us":108,"threads_started":1}
I20260806 21:06:23.042639 29605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:23.035359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58946 (local address 127.27.225.194:44153)
0806 21:06:23.035890 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:23.035903 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:23.036309 (+ 406us) server_negotiation.cc:407] Connection header received
0806 21:06:23.036397 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:23.036403 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:23.036611 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:23.036759 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:23.037281 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.038543 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.039981 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.040868 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.040997 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:23.041135 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:23.042128 (+ 993us) server_negotiation.cc:299] Negotiation successful
0806 21:06:23.042494 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":112,"threads_started":1}
I20260806 21:06:23.045780 29254 tablet_service.cc:1467] Processing CreateTablet for tablet f80fd5d148454367a9673ccb39ac10e0 (DEFAULT_TABLE table=test-table [id=04f8f766d0354c17bdf221ff132e084a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:23.045964 29390 tablet_service.cc:1467] Processing CreateTablet for tablet f80fd5d148454367a9673ccb39ac10e0 (DEFAULT_TABLE table=test-table [id=04f8f766d0354c17bdf221ff132e084a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:23.047254 29254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f80fd5d148454367a9673ccb39ac10e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:23.048022 29525 tablet_service.cc:1467] Processing CreateTablet for tablet f80fd5d148454367a9673ccb39ac10e0 (DEFAULT_TABLE table=test-table [id=04f8f766d0354c17bdf221ff132e084a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:23.049088 29525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f80fd5d148454367a9673ccb39ac10e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:23.049082 29390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f80fd5d148454367a9673ccb39ac10e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:23.055655 29609 tablet_bootstrap.cc:492] T f80fd5d148454367a9673ccb39ac10e0 P b8e88404f0b04b5a9efe2875af0204e5: Bootstrap starting.
I20260806 21:06:23.056041 29610 tablet_bootstrap.cc:492] T f80fd5d148454367a9673ccb39ac10e0 P ffa112f475c542d0a0df31a04ba9cd33: Bootstrap starting.
I20260806 21:06:23.056970 29611 tablet_bootstrap.cc:492] T f80fd5d148454367a9673ccb39ac10e0 P 84e782d1156b4aa8a88391410a8ca17a: Bootstrap starting.
I20260806 21:06:23.059170 29609 tablet_bootstrap.cc:654] T f80fd5d148454367a9673ccb39ac10e0 P b8e88404f0b04b5a9efe2875af0204e5: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:23.059476 29610 tablet_bootstrap.cc:654] T f80fd5d148454367a9673ccb39ac10e0 P ffa112f475c542d0a0df31a04ba9cd33: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:23.059883 29611 tablet_bootstrap.cc:654] T f80fd5d148454367a9673ccb39ac10e0 P 84e782d1156b4aa8a88391410a8ca17a: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:23.060545 29610 log.cc:824] T f80fd5d148454367a9673ccb39ac10e0 P ffa112f475c542d0a0df31a04ba9cd33: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:23.060545 29609 log.cc:824] T f80fd5d148454367a9673ccb39ac10e0 P b8e88404f0b04b5a9efe2875af0204e5: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:23.060570 29611 log.cc:824] T f80fd5d148454367a9673ccb39ac10e0 P 84e782d1156b4aa8a88391410a8ca17a: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:23.062536 29610 tablet_bootstrap.cc:492] T f80fd5d148454367a9673ccb39ac10e0 P ffa112f475c542d0a0df31a04ba9cd33: No bootstrap required, opened a new log
I20260806 21:06:23.062624 29611 tablet_bootstrap.cc:492] T f80fd5d148454367a9673ccb39ac10e0 P 84e782d1156b4aa8a88391410a8ca17a: No bootstrap required, opened a new log
I20260806 21:06:23.062644 29609 tablet_bootstrap.cc:492] T f80fd5d148454367a9673ccb39ac10e0 P b8e88404f0b04b5a9efe2875af0204e5: No bootstrap required, opened a new log
I20260806 21:06:23.062798 29610 ts_tablet_manager.cc:1397] T f80fd5d148454367a9673ccb39ac10e0 P ffa112f475c542d0a0df31a04ba9cd33: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 21:06:23.062820 29609 ts_tablet_manager.cc:1397] T f80fd5d148454367a9673ccb39ac10e0 P b8e88404f0b04b5a9efe2875af0204e5: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 21:06:23.062820 29611 ts_tablet_manager.cc:1397] T f80fd5d148454367a9673ccb39ac10e0 P 84e782d1156b4aa8a88391410a8ca17a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 21:06:23.066537 29610 raft_consensus.cc:348] T f80fd5d148454367a9673ccb39ac10e0 P ffa112f475c542d0a0df31a04ba9cd33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffa112f475c542d0a0df31a04ba9cd33" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 36509 } } peers { permanent_uuid: "84e782d1156b4aa8a88391410a8ca17a" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 44153 } } peers { permanent_uuid: "b8e88404f0b04b5a9efe2875af0204e5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34321 } }
I20260806 21:06:23.066809 29610 raft_consensus.cc:374] T f80fd5d148454367a9673ccb39ac10e0 P ffa112f475c542d0a0df31a04ba9cd33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:23.066862 29610 raft_consensus.cc:729] T f80fd5d148454367a9673ccb39ac10e0 P ffa112f475c542d0a0df31a04ba9cd33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffa112f475c542d0a0df31a04ba9cd33, State: Initialized, Role: FOLLOWER
I20260806 21:06:23.066717 29611 raft_consensus.cc:348] T f80fd5d148454367a9673ccb39ac10e0 P 84e782d1156b4aa8a88391410a8ca17a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffa112f475c542d0a0df31a04ba9cd33" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 36509 } } peers { permanent_uuid: "84e782d1156b4aa8a88391410a8ca17a" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 44153 } } peers { permanent_uuid: "b8e88404f0b04b5a9efe2875af0204e5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34321 } }
I20260806 21:06:23.067000 29611 raft_consensus.cc:374] T f80fd5d148454367a9673ccb39ac10e0 P 84e782d1156b4aa8a88391410a8ca17a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:23.067063 29611 raft_consensus.cc:729] T f80fd5d148454367a9673ccb39ac10e0 P 84e782d1156b4aa8a88391410a8ca17a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84e782d1156b4aa8a88391410a8ca17a, State: Initialized, Role: FOLLOWER
I20260806 21:06:23.067358 29610 consensus_queue.cc:260] T f80fd5d148454367a9673ccb39ac10e0 P ffa112f475c542d0a0df31a04ba9cd33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffa112f475c542d0a0df31a04ba9cd33" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 36509 } } peers { permanent_uuid: "84e782d1156b4aa8a88391410a8ca17a" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 44153 } } peers { permanent_uuid: "b8e88404f0b04b5a9efe2875af0204e5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34321 } }
I20260806 21:06:23.067375 29609 raft_consensus.cc:348] T f80fd5d148454367a9673ccb39ac10e0 P b8e88404f0b04b5a9efe2875af0204e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffa112f475c542d0a0df31a04ba9cd33" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 36509 } } peers { permanent_uuid: "84e782d1156b4aa8a88391410a8ca17a" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 44153 } } peers { permanent_uuid: "b8e88404f0b04b5a9efe2875af0204e5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34321 } }
I20260806 21:06:23.067549 29611 consensus_queue.cc:260] T f80fd5d148454367a9673ccb39ac10e0 P 84e782d1156b4aa8a88391410a8ca17a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffa112f475c542d0a0df31a04ba9cd33" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 36509 } } peers { permanent_uuid: "84e782d1156b4aa8a88391410a8ca17a" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 44153 } } peers { permanent_uuid: "b8e88404f0b04b5a9efe2875af0204e5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34321 } }
I20260806 21:06:23.067652 29609 raft_consensus.cc:374] T f80fd5d148454367a9673ccb39ac10e0 P b8e88404f0b04b5a9efe2875af0204e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:23.067703 29609 raft_consensus.cc:729] T f80fd5d148454367a9673ccb39ac10e0 P b8e88404f0b04b5a9efe2875af0204e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e88404f0b04b5a9efe2875af0204e5, State: Initialized, Role: FOLLOWER
I20260806 21:06:23.068141 29609 consensus_queue.cc:260] T f80fd5d148454367a9673ccb39ac10e0 P b8e88404f0b04b5a9efe2875af0204e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffa112f475c542d0a0df31a04ba9cd33" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 36509 } } peers { permanent_uuid: "84e782d1156b4aa8a88391410a8ca17a" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 44153 } } peers { permanent_uuid: "b8e88404f0b04b5a9efe2875af0204e5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34321 } }
I20260806 21:06:23.068284 29321 heartbeater.cc:503] Master 127.27.225.254:36279 was elected leader, sending a full tablet report...
I20260806 21:06:23.068535 29457 heartbeater.cc:503] Master 127.27.225.254:36279 was elected leader, sending a full tablet report...
I20260806 21:06:23.068531 29610 ts_tablet_manager.cc:1428] T f80fd5d148454367a9673ccb39ac10e0 P ffa112f475c542d0a0df31a04ba9cd33: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 21:06:23.069167 29609 ts_tablet_manager.cc:1428] T f80fd5d148454367a9673ccb39ac10e0 P b8e88404f0b04b5a9efe2875af0204e5: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260806 21:06:23.069207 29611 ts_tablet_manager.cc:1428] T f80fd5d148454367a9673ccb39ac10e0 P 84e782d1156b4aa8a88391410a8ca17a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 21:06:23.069571 29591 heartbeater.cc:503] Master 127.27.225.254:36279 was elected leader, sending a full tablet report...
I20260806 21:06:23.199326 29615 raft_consensus.cc:482] T f80fd5d148454367a9673ccb39ac10e0 P ffa112f475c542d0a0df31a04ba9cd33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:06:23.199570 29615 raft_consensus.cc:504] T f80fd5d148454367a9673ccb39ac10e0 P ffa112f475c542d0a0df31a04ba9cd33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffa112f475c542d0a0df31a04ba9cd33" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 36509 } } peers { permanent_uuid: "84e782d1156b4aa8a88391410a8ca17a" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 44153 } } peers { permanent_uuid: "b8e88404f0b04b5a9efe2875af0204e5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34321 } }
I20260806 21:06:23.200763 29615 leader_election.cc:288] T f80fd5d148454367a9673ccb39ac10e0 P ffa112f475c542d0a0df31a04ba9cd33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84e782d1156b4aa8a88391410a8ca17a (127.27.225.194:44153), b8e88404f0b04b5a9efe2875af0204e5 (127.27.225.195:34321)
I20260806 21:06:23.206879 29605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:23.201209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:42289 (local address 127.27.225.194:44153)
0806 21:06:23.201346 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:23.201351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:23.201689 (+ 338us) server_negotiation.cc:407] Connection header received
0806 21:06:23.201897 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:23.201901 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:23.201959 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:23.202045 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:23.202542 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.203348 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.205295 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.205998 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.206115 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:23.206170 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:23.206654 (+ 484us) server_negotiation.cc:299] Negotiation successful
0806 21:06:23.206737 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 21:06:23.206879 29608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:23.201221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:41849 (local address 127.27.225.195:34321)
0806 21:06:23.201346 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:23.201351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:23.201680 (+ 329us) server_negotiation.cc:407] Connection header received
0806 21:06:23.201898 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:23.201901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:23.201957 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:23.202045 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:23.202542 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.203349 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.205295 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.205978 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.206115 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:23.206182 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:23.206654 (+ 472us) server_negotiation.cc:299] Negotiation successful
0806 21:06:23.206758 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 21:06:23.207027 29620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:23.201107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:34321 (local address 127.27.225.193:41849)
0806 21:06:23.201572 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:23.201585 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:23.201740 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:23.202067 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:23.202070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:23.202095 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:23.202376 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:23.202384 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.203478 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.203482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:23.205156 (+ 1674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:23.205168 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.206120 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.206124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:23.206405 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:23.206523 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 21:06:23.206609 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 21:06:23.206807 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"mutex_wait_us":401,"spinlock_wait_cycles":22528,"thread_start_us":105,"threads_started":1}
I20260806 21:06:23.207217 29621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:23.201107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:44153 (local address 127.27.225.193:42289)
0806 21:06:23.201545 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:23.201561 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:23.201740 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:23.202067 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:23.202070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:23.202093 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:23.202376 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:23.202384 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.203478 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.203482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:23.205157 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:23.205168 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.206120 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.206124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:23.206433 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:23.206528 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 21:06:23.206609 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 21:06:23.206807 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":459,"spinlock_wait_cycles":26880,"thread_start_us":161,"threads_started":1}
I20260806 21:06:23.207880 29545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f80fd5d148454367a9673ccb39ac10e0" candidate_uuid: "ffa112f475c542d0a0df31a04ba9cd33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e88404f0b04b5a9efe2875af0204e5" is_pre_election: true
I20260806 21:06:23.207891 29410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f80fd5d148454367a9673ccb39ac10e0" candidate_uuid: "ffa112f475c542d0a0df31a04ba9cd33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84e782d1156b4aa8a88391410a8ca17a" is_pre_election: true
I20260806 21:06:23.208269 29410 raft_consensus.cc:2454] T f80fd5d148454367a9673ccb39ac10e0 P 84e782d1156b4aa8a88391410a8ca17a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffa112f475c542d0a0df31a04ba9cd33 in term 0.
I20260806 21:06:23.208290 29545 raft_consensus.cc:2454] T f80fd5d148454367a9673ccb39ac10e0 P b8e88404f0b04b5a9efe2875af0204e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffa112f475c542d0a0df31a04ba9cd33 in term 0.
I20260806 21:06:23.208874 29207 leader_election.cc:302] T f80fd5d148454367a9673ccb39ac10e0 P ffa112f475c542d0a0df31a04ba9cd33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8e88404f0b04b5a9efe2875af0204e5, ffa112f475c542d0a0df31a04ba9cd33; no voters:
I20260806 21:06:23.209125 29615 raft_consensus.cc:2781] T f80fd5d148454367a9673ccb39ac10e0 P ffa112f475c542d0a0df31a04ba9cd33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:06:23.209263 29615 raft_consensus.cc:482] T f80fd5d148454367a9673ccb39ac10e0 P ffa112f475c542d0a0df31a04ba9cd33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:06:23.209319 29615 raft_consensus.cc:3037] T f80fd5d148454367a9673ccb39ac10e0 P ffa112f475c542d0a0df31a04ba9cd33 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:23.210495 29615 raft_consensus.cc:504] T f80fd5d148454367a9673ccb39ac10e0 P ffa112f475c542d0a0df31a04ba9cd33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffa112f475c542d0a0df31a04ba9cd33" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 36509 } } peers { permanent_uuid: "84e782d1156b4aa8a88391410a8ca17a" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 44153 } } peers { permanent_uuid: "b8e88404f0b04b5a9efe2875af0204e5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34321 } }
I20260806 21:06:23.210873 29615 leader_election.cc:288] T f80fd5d148454367a9673ccb39ac10e0 P ffa112f475c542d0a0df31a04ba9cd33 [CANDIDATE]: Term 1 election: Requested vote from peers 84e782d1156b4aa8a88391410a8ca17a (127.27.225.194:44153), b8e88404f0b04b5a9efe2875af0204e5 (127.27.225.195:34321)
I20260806 21:06:23.211302 29545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f80fd5d148454367a9673ccb39ac10e0" candidate_uuid: "ffa112f475c542d0a0df31a04ba9cd33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e88404f0b04b5a9efe2875af0204e5"
I20260806 21:06:23.211349 29410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f80fd5d148454367a9673ccb39ac10e0" candidate_uuid: "ffa112f475c542d0a0df31a04ba9cd33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84e782d1156b4aa8a88391410a8ca17a"
I20260806 21:06:23.211436 29545 raft_consensus.cc:3037] T f80fd5d148454367a9673ccb39ac10e0 P b8e88404f0b04b5a9efe2875af0204e5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:23.211542 29410 raft_consensus.cc:3037] T f80fd5d148454367a9673ccb39ac10e0 P 84e782d1156b4aa8a88391410a8ca17a [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:23.212615 29545 raft_consensus.cc:2454] T f80fd5d148454367a9673ccb39ac10e0 P b8e88404f0b04b5a9efe2875af0204e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffa112f475c542d0a0df31a04ba9cd33 in term 1.
I20260806 21:06:23.212687 29410 raft_consensus.cc:2454] T f80fd5d148454367a9673ccb39ac10e0 P 84e782d1156b4aa8a88391410a8ca17a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffa112f475c542d0a0df31a04ba9cd33 in term 1.
I20260806 21:06:23.213043 29208 leader_election.cc:302] T f80fd5d148454367a9673ccb39ac10e0 P ffa112f475c542d0a0df31a04ba9cd33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84e782d1156b4aa8a88391410a8ca17a, ffa112f475c542d0a0df31a04ba9cd33; no voters:
I20260806 21:06:23.213248 29615 raft_consensus.cc:2781] T f80fd5d148454367a9673ccb39ac10e0 P ffa112f475c542d0a0df31a04ba9cd33 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:23.213518 29615 raft_consensus.cc:686] T f80fd5d148454367a9673ccb39ac10e0 P ffa112f475c542d0a0df31a04ba9cd33 [term 1 LEADER]: Becoming Leader. State: Replica: ffa112f475c542d0a0df31a04ba9cd33, State: Running, Role: LEADER
W20260806 21:06:23.213955 29592 tablet.cc:2369] T f80fd5d148454367a9673ccb39ac10e0 P b8e88404f0b04b5a9efe2875af0204e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:06:23.213933 29615 consensus_queue.cc:237] T f80fd5d148454367a9673ccb39ac10e0 P ffa112f475c542d0a0df31a04ba9cd33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffa112f475c542d0a0df31a04ba9cd33" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 36509 } } peers { permanent_uuid: "84e782d1156b4aa8a88391410a8ca17a" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 44153 } } peers { permanent_uuid: "b8e88404f0b04b5a9efe2875af0204e5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34321 } }
I20260806 21:06:23.217391 29127 catalog_manager.cc:5495] T f80fd5d148454367a9673ccb39ac10e0 P ffa112f475c542d0a0df31a04ba9cd33 reported cstate change: term changed from 0 to 1, leader changed from <none> to ffa112f475c542d0a0df31a04ba9cd33 (127.27.225.193). New cstate: current_term: 1 leader_uuid: "ffa112f475c542d0a0df31a04ba9cd33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffa112f475c542d0a0df31a04ba9cd33" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 36509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84e782d1156b4aa8a88391410a8ca17a" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 44153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8e88404f0b04b5a9efe2875af0204e5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34321 } health_report { overall_health: UNKNOWN } } }
W20260806 21:06:23.228164 29322 tablet.cc:2369] T f80fd5d148454367a9673ccb39ac10e0 P ffa112f475c542d0a0df31a04ba9cd33: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 21:06:23.234963 29458 tablet.cc:2369] T f80fd5d148454367a9673ccb39ac10e0 P 84e782d1156b4aa8a88391410a8ca17a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 21:06:23 dist-test-slave-69rg krb5kdc[29080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.193@KRBTEST.COM
I20260806 21:06:23.241858 29607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:23.227915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60544 (local address 127.27.225.193:36509)
0806 21:06:23.228100 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:23.228106 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:23.228243 (+ 137us) server_negotiation.cc:407] Connection header received
0806 21:06:23.228471 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:23.228474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:23.228527 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:23.228622 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:23.229450 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.230122 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.231132 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.231382 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.231502 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:23.238197 (+ 6695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:23.238218 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:23.238240 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:23.238274 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:23.239916 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:23.240431 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:23.240437 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:23.240438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:23.240535 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:23.240828 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:23.240832 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:23.240834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:23.241102 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:23.241322 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:23.241512 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 21:06:23.241647 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
Aug 06 21:06:23 dist-test-slave-69rg krb5kdc[29080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050383, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 21:06:23 dist-test-slave-69rg krb5kdc[29080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050383, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.225.193@KRBTEST.COM
I20260806 21:06:23.274394 29607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:23.263930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60556 (local address 127.27.225.193:36509)
0806 21:06:23.264131 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:23.264136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:23.264228 (+ 92us) server_negotiation.cc:407] Connection header received
0806 21:06:23.264417 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:23.264421 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:23.264475 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:23.264554 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:23.265349 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.265949 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.267035 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.267313 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.267422 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:23.270673 (+ 3251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:23.270677 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:23.270679 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:23.270710 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:23.272257 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:23.272776 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:23.272791 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:23.272792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:23.272875 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:23.273183 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:23.273186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:23.273187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:23.273387 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:23.273507 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:23.273810 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 21:06:23.274267 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
Aug 06 21:06:23 dist-test-slave-69rg krb5kdc[29080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 21:06:23 dist-test-slave-69rg krb5kdc[29080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.193@KRBTEST.COM
I20260806 21:06:23.306331 29607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:23.295662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60562 (local address 127.27.225.193:36509)
0806 21:06:23.295885 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:23.295891 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:23.296052 (+ 161us) server_negotiation.cc:407] Connection header received
0806 21:06:23.296296 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:23.296299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:23.296349 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:23.296461 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:23.297371 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.298111 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.299056 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:23.299320 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:23.299439 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:23.302399 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:23.302403 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:23.302406 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:23.302439 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:23.304121 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:23.304778 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:23.304782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:23.304784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:23.304876 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:23.305230 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:23.305234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:23.305239 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:23.305486 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:23.305643 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:23.305963 (+ 320us) server_negotiation.cc:299] Negotiation successful
0806 21:06:23.306113 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
W20260806 21:06:23.308355 29234 tablet_service.cc:2908] Rejecting scan request for tablet f80fd5d148454367a9673ccb39ac10e0: Uninitialized: clean time has not yet been initialized
I20260806 21:06:23.311570 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 29190
I20260806 21:06:23.319778 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 29326
I20260806 21:06:23.327024 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 29462
I20260806 21:06:23.334558 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 29095
2026-08-06T21:06:23Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2673 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:06:23 dist-test-slave-69rg krb5kdc[29637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:06:23 dist-test-slave-69rg krb5kdc[29637](info): set up 2 sockets
Aug 06 21:06:23 dist-test-slave-69rg krb5kdc[29637](info): commencing operation
krb5kdc: starting...
W20260806 21:06:24.396986 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:06:24 dist-test-slave-69rg krb5kdc[29637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:06:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:06:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:06:24.534160 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:33913
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:44557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:33913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:24.651068 29651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:24.651417 29651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:24.651464 29651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:24.655939 29651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:06:24.656029 29651 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 21:06:24.656047 29651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:24.656069 29651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:24.656085 29651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:06:24.656101 29651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:06:24.662250 29651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:44557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:33913
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:33913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:24.663434 29651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:24.664486 29651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:24.675742 29656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:24.675755 29659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:24.675808 29651 server_base.cc:1034] running on GCE node
W20260806 21:06:24.675843 29657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:24.676765 29651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:24.678143 29651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:24.679335 29651 hybrid_clock.cc:648] HybridClock initialized: now 1786050384679312 us; error 34 us; skew 500 ppm
Aug 06 21:06:24 dist-test-slave-69rg krb5kdc[29637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050384, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:24.682948 29651 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:06:24.684059 29651 webserver.cc:460] Webserver started at http://127.27.225.254:41953/ using document root <none> and password file <none>
I20260806 21:06:24.684710 29651 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:24.684782 29651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:24.685022 29651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:24.687422 29651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "2366e370d4a040c7b31d453c341abe4c"
format_stamp: "Formatted at 2026-08-06 21:06:24 on dist-test-slave-69rg"
server_key: "97de89c6b2e21dba0867112147b4f0a4"
server_key_iv: "a64bc028ab83a02b940773393b57c3b5"
server_key_version: "encryptionkey@0"
I20260806 21:06:24.688069 29651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "2366e370d4a040c7b31d453c341abe4c"
format_stamp: "Formatted at 2026-08-06 21:06:24 on dist-test-slave-69rg"
server_key: "97de89c6b2e21dba0867112147b4f0a4"
server_key_iv: "a64bc028ab83a02b940773393b57c3b5"
server_key_version: "encryptionkey@0"
I20260806 21:06:24.692459 29651 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:06:24.695479 29666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:24.697016 29651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 21:06:24.697222 29651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "2366e370d4a040c7b31d453c341abe4c"
format_stamp: "Formatted at 2026-08-06 21:06:24 on dist-test-slave-69rg"
server_key: "97de89c6b2e21dba0867112147b4f0a4"
server_key_iv: "a64bc028ab83a02b940773393b57c3b5"
server_key_version: "encryptionkey@0"
I20260806 21:06:24.697399 29651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:24.709908 29651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:24.713902 29651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:24.714159 29651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:24.723882 29651 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:33913
I20260806 21:06:24.723901 29729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:33913 every 8 connection(s)
I20260806 21:06:24.725005 29651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:06:24.728741 29730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:24.732678 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 29651
I20260806 21:06:24.732797 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:06:24.733081 28551 external_mini_cluster.cc:1428] Setting key bdf4a3ec98c83790224d3b0b6d9eda8e
I20260806 21:06:24.736485 29730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2366e370d4a040c7b31d453c341abe4c: Bootstrap starting.
I20260806 21:06:24.739074 29730 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2366e370d4a040c7b31d453c341abe4c: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:24.739926 29730 log.cc:824] T 00000000000000000000000000000000 P 2366e370d4a040c7b31d453c341abe4c: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:24.742141 29730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2366e370d4a040c7b31d453c341abe4c: No bootstrap required, opened a new log
Aug 06 21:06:24 dist-test-slave-69rg krb5kdc[29637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:24.746762 29730 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2366e370d4a040c7b31d453c341abe4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2366e370d4a040c7b31d453c341abe4c" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 33913 } }
I20260806 21:06:24.747125 29730 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2366e370d4a040c7b31d453c341abe4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:24.747206 29730 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2366e370d4a040c7b31d453c341abe4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2366e370d4a040c7b31d453c341abe4c, State: Initialized, Role: FOLLOWER
I20260806 21:06:24.747694 29730 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2366e370d4a040c7b31d453c341abe4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2366e370d4a040c7b31d453c341abe4c" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 33913 } }
I20260806 21:06:24.747839 29730 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2366e370d4a040c7b31d453c341abe4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:06:24.747943 29730 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2366e370d4a040c7b31d453c341abe4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:06:24.748070 29730 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2366e370d4a040c7b31d453c341abe4c [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:24.749114 29730 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2366e370d4a040c7b31d453c341abe4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2366e370d4a040c7b31d453c341abe4c" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 33913 } }
I20260806 21:06:24.749512 29730 leader_election.cc:302] T 00000000000000000000000000000000 P 2366e370d4a040c7b31d453c341abe4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2366e370d4a040c7b31d453c341abe4c; no voters:
I20260806 21:06:24.749922 29730 leader_election.cc:288] T 00000000000000000000000000000000 P 2366e370d4a040c7b31d453c341abe4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:06:24.750042 29735 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2366e370d4a040c7b31d453c341abe4c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:24.750365 29735 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2366e370d4a040c7b31d453c341abe4c [term 1 LEADER]: Becoming Leader. State: Replica: 2366e370d4a040c7b31d453c341abe4c, State: Running, Role: LEADER
I20260806 21:06:24.750761 29735 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2366e370d4a040c7b31d453c341abe4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2366e370d4a040c7b31d453c341abe4c" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 33913 } }
I20260806 21:06:24.751709 29730 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2366e370d4a040c7b31d453c341abe4c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:06:24.752245 29735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2366e370d4a040c7b31d453c341abe4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2366e370d4a040c7b31d453c341abe4c. Latest consensus state: current_term: 1 leader_uuid: "2366e370d4a040c7b31d453c341abe4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2366e370d4a040c7b31d453c341abe4c" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 33913 } } }
I20260806 21:06:24.752401 29735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2366e370d4a040c7b31d453c341abe4c [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:24.752681 29736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2366e370d4a040c7b31d453c341abe4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2366e370d4a040c7b31d453c341abe4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2366e370d4a040c7b31d453c341abe4c" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 33913 } } }
I20260806 21:06:24.752797 29736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2366e370d4a040c7b31d453c341abe4c [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:24.753093 29733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:24.735168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50776 (local address 127.27.225.254:33913)
0806 21:06:24.735882 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:24.735899 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:24.735939 (+ 40us) server_negotiation.cc:407] Connection header received
0806 21:06:24.736942 (+ 1003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:24.736968 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:24.737435 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:24.737877 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:24.739065 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:24.740390 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:24.741433 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:24.742128 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:24.742401 (+ 273us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:24.745665 (+ 3264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:24.745672 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:24.745691 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:24.745732 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:24.748344 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:24.748968 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:24.748979 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:24.748986 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:24.749131 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:24.749475 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:24.749480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:24.749482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:24.750184 (+ 702us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:24.751308 (+ 1124us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:24.751491 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 21:06:24.751862 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":211,"threads_started":1}
I20260806 21:06:24.753794 29740 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:06:24.758370 29740 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:06:24.766167 29740 catalog_manager.cc:1347] Generated new cluster ID: 42170c6239794690ae1fd12024d2899c
I20260806 21:06:24.766301 29740 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:06:24.776818 29740 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:06:24.778448 29740 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:06:24.784178 29740 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2366e370d4a040c7b31d453c341abe4c: Generated new TSK 0
I20260806 21:06:24.785399 29740 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.225.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
I20260806 21:06:24.833747 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:33913
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:44557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:24.975651 29757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:24.975994 29757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:24.976048 29757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:24.980854 29757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:24.980959 29757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:24.981002 29757 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 21:06:24.981037 29757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:06:24.987210 29757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:44557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.225.254:33913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:24.988444 29757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:24.989547 29757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:25.001515 29763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:25.001559 29765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:25.001492 29762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:25.003103 29757 server_base.cc:1034] running on GCE node
I20260806 21:06:25.003918 29757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:25.004817 29757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:25.006064 29757 hybrid_clock.cc:648] HybridClock initialized: now 1786050385006032 us; error 54 us; skew 500 ppm
Aug 06 21:06:25 dist-test-slave-69rg krb5kdc[29637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050385, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:25.009665 29757 init.cc:377] Logged in from keytab as kudu/127.27.225.193@KRBTEST.COM (short username kudu)
I20260806 21:06:25.010882 29757 webserver.cc:460] Webserver started at http://127.27.225.193:42417/ using document root <none> and password file <none>
I20260806 21:06:25.011554 29757 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:25.011662 29757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:25.011912 29757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:25.015275 29757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "b2c7539b602c4861a114b8912e36d1de"
format_stamp: "Formatted at 2026-08-06 21:06:25 on dist-test-slave-69rg"
server_key: "5e3f6771b0a938074e275b59b7ac6fa4"
server_key_iv: "7925151fdca084025761a011b0dd804e"
server_key_version: "encryptionkey@0"
I20260806 21:06:25.016374 29757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "b2c7539b602c4861a114b8912e36d1de"
format_stamp: "Formatted at 2026-08-06 21:06:25 on dist-test-slave-69rg"
server_key: "5e3f6771b0a938074e275b59b7ac6fa4"
server_key_iv: "7925151fdca084025761a011b0dd804e"
server_key_version: "encryptionkey@0"
I20260806 21:06:25.022799 29757 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260806 21:06:25.026751 29772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:25.028414 29757 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 21:06:25.028580 29757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "b2c7539b602c4861a114b8912e36d1de"
format_stamp: "Formatted at 2026-08-06 21:06:25 on dist-test-slave-69rg"
server_key: "5e3f6771b0a938074e275b59b7ac6fa4"
server_key_iv: "7925151fdca084025761a011b0dd804e"
server_key_version: "encryptionkey@0"
I20260806 21:06:25.028712 29757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:25.052985 29757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:25.057057 29757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:25.057261 29757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:25.059280 29757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:25.059350 29757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:25.059420 29757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:25.059453 29757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 21:06:25 dist-test-slave-69rg krb5kdc[29637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050385, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:25.071640 29757 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:37399
I20260806 21:06:25.071803 29891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:37399 every 8 connection(s)
I20260806 21:06:25.073000 29757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:06:25.074342 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 29757
I20260806 21:06:25.074493 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:06:25.074802 28551 external_mini_cluster.cc:1428] Setting key 74154d5b9a83122d640d71739d86458e
I20260806 21:06:25.076143 29733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:25.060050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:59087 (local address 127.27.225.254:33913)
0806 21:06:25.060221 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:25.060226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:25.062189 (+ 1963us) server_negotiation.cc:407] Connection header received
0806 21:06:25.063662 (+ 1473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:25.063666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:25.063772 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:25.063888 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:25.065924 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.066488 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.067788 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.068056 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.068177 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:25.071186 (+ 3009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:25.071191 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:25.071195 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:25.071235 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:25.073243 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:25.074134 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:25.074138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:25.074140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:25.074270 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:25.074753 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:25.074757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:25.074759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:25.075033 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:25.075146 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:25.075808 (+ 662us) server_negotiation.cc:299] Negotiation successful
0806 21:06:25.075964 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 21:06:25.077088 29782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:25.061249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:33913 (local address 127.27.225.193:59087)
0806 21:06:25.061980 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:25.062046 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:25.063367 (+ 1321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:25.064140 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:25.064152 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:25.064780 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:25.065668 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:25.065708 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.066649 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.066653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:25.067627 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:25.067638 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.068254 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.068258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:25.068451 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:25.069147 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:25.069165 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:25.070938 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:25.073424 (+ 2486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:25.073431 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:25.073450 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:25.073967 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:25.074386 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:25.074390 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:25.074392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:25.074614 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:25.075163 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:25.075171 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:25.075510 (+ 339us) client_negotiation.cc:769] Sending connection context
0806 21:06:25.075743 (+ 233us) client_negotiation.cc:240] Negotiation successful
0806 21:06:25.076106 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":135,"threads_started":1}
I20260806 21:06:25.078510 29892 heartbeater.cc:347] Connected to a master server at 127.27.225.254:33913
I20260806 21:06:25.078708 29892 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:25.079265 29892 heartbeater.cc:511] Master 127.27.225.254:33913 requested a full tablet report, sending...
I20260806 21:06:25.081233 29683 ts_manager.cc:194] Registered new tserver with Master: b2c7539b602c4861a114b8912e36d1de (127.27.225.193:37399)
I20260806 21:06:25.082602 29683 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.193@KRBTEST.COM'} at 127.27.225.193:59087
WARNING: no policy specified for kudu/127.27.225.194@KRBTEST.COM; defaulting to no policy
I20260806 21:06:25.093243 29733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:25.083337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:48765 (local address 127.27.225.254:33913)
0806 21:06:25.083524 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:25.083529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:25.083544 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:06:25.083583 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:25.083585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:25.083661 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:25.083783 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:25.084454 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.085270 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.086739 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.087006 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.087107 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:25.088756 (+ 1649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:25.088760 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:25.088762 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:25.088793 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:25.090961 (+ 2168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:25.091522 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:25.091526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:25.091527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:25.091676 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:25.092054 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:25.092058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:25.092060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:25.092354 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:25.092491 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:25.092873 (+ 382us) server_negotiation.cc:299] Negotiation successful
0806 21:06:25.093032 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 21:06:25.093251 29899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:25.081061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:33913 (local address 127.27.225.193:48765)
0806 21:06:25.081451 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:25.081468 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:25.081589 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:25.083805 (+ 2216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:25.083812 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:25.084101 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:25.084276 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:25.084288 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.085452 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.085456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:25.086554 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:25.086584 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.087149 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.087155 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:25.087299 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:25.087937 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:25.087954 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:25.088473 (+ 519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:25.091121 (+ 2648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:25.091125 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:25.091126 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:25.091381 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:25.091814 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:25.091818 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:25.091820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:25.091925 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:25.092481 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:25.092484 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:25.092610 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 21:06:25.092799 (+ 189us) client_negotiation.cc:240] Negotiation successful
0806 21:06:25.093016 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":154,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
I20260806 21:06:25.117569 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:33913
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:44557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:25.251787 29901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:25.252151 29901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:25.252255 29901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:25.257210 29901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:25.257333 29901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:25.257409 29901 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 21:06:25.257473 29901 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:06:25.264335 29901 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:44557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.225.254:33913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:25.265681 29901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:25.266898 29901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:25.277451 29910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:25.277568 29907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:25.277618 29901 server_base.cc:1034] running on GCE node
W20260806 21:06:25.277568 29908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:25.278355 29901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:25.279248 29901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:25.280509 29901 hybrid_clock.cc:648] HybridClock initialized: now 1786050385280463 us; error 62 us; skew 500 ppm
Aug 06 21:06:25 dist-test-slave-69rg krb5kdc[29637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050385, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:25.284176 29901 init.cc:377] Logged in from keytab as kudu/127.27.225.194@KRBTEST.COM (short username kudu)
I20260806 21:06:25.285353 29901 webserver.cc:460] Webserver started at http://127.27.225.194:43659/ using document root <none> and password file <none>
I20260806 21:06:25.286003 29901 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:25.286094 29901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:25.286281 29901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:25.288530 29901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "1a90a8480fa14f169d39d639797882e0"
format_stamp: "Formatted at 2026-08-06 21:06:25 on dist-test-slave-69rg"
server_key: "7a099ccd6bedd0c90675f00addc9e163"
server_key_iv: "8a351f4bf45ccf5433b784ad5d0b1a40"
server_key_version: "encryptionkey@0"
I20260806 21:06:25.289186 29901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "1a90a8480fa14f169d39d639797882e0"
format_stamp: "Formatted at 2026-08-06 21:06:25 on dist-test-slave-69rg"
server_key: "7a099ccd6bedd0c90675f00addc9e163"
server_key_iv: "8a351f4bf45ccf5433b784ad5d0b1a40"
server_key_version: "encryptionkey@0"
I20260806 21:06:25.293689 29901 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:06:25.297533 29917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:25.299346 29901 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 21:06:25.299507 29901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "1a90a8480fa14f169d39d639797882e0"
format_stamp: "Formatted at 2026-08-06 21:06:25 on dist-test-slave-69rg"
server_key: "7a099ccd6bedd0c90675f00addc9e163"
server_key_iv: "8a351f4bf45ccf5433b784ad5d0b1a40"
server_key_version: "encryptionkey@0"
I20260806 21:06:25.299718 29901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:25.323850 29901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:25.328823 29901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:25.329082 29901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:25.330978 29901 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:25.331044 29901 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:25.331089 29901 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:25.331164 29901 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 21:06:25 dist-test-slave-69rg krb5kdc[29637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050385, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:25.344913 29901 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:39377
I20260806 21:06:25.345194 30035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:39377 every 8 connection(s)
I20260806 21:06:25.346279 29901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:06:25.347786 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 29901
I20260806 21:06:25.347914 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:06:25.348304 28551 external_mini_cluster.cc:1428] Setting key 5023b6e741c7fae32c5fda20f7e3cb49
I20260806 21:06:25.348556 29733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:25.332249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:38629 (local address 127.27.225.254:33913)
0806 21:06:25.332432 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:25.332438 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:25.333614 (+ 1176us) server_negotiation.cc:407] Connection header received
0806 21:06:25.335309 (+ 1695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:25.335313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:25.335391 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:25.335492 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:25.337704 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.338441 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.339981 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.340279 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.340408 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:25.343705 (+ 3297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:25.343710 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:25.343713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:25.343754 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:25.345885 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:25.346527 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:25.346532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:25.346535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:25.346671 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:25.347131 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:25.347135 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:25.347136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:25.347401 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:25.347489 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:25.348260 (+ 771us) server_negotiation.cc:299] Negotiation successful
0806 21:06:25.348421 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 21:06:25.349673 29926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:25.332792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:33913 (local address 127.27.225.194:38629)
0806 21:06:25.333408 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:25.333471 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:25.334981 (+ 1510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:25.335876 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:25.335884 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:25.336540 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:25.337420 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:25.337446 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.338706 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.338709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:25.339823 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:25.339834 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.340475 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.340479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:25.340677 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:25.341491 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:25.341509 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:25.343415 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:25.346033 (+ 2618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:25.346038 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:25.346051 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:25.346420 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:25.346819 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:25.346823 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:25.346825 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:25.347003 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:25.347540 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:25.347547 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:25.347939 (+ 392us) client_negotiation.cc:769] Sending connection context
0806 21:06:25.348199 (+ 260us) client_negotiation.cc:240] Negotiation successful
0806 21:06:25.348648 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":181,"threads_started":1}
I20260806 21:06:25.351042 30036 heartbeater.cc:347] Connected to a master server at 127.27.225.254:33913
I20260806 21:06:25.351250 30036 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:25.351909 30036 heartbeater.cc:511] Master 127.27.225.254:33913 requested a full tablet report, sending...
I20260806 21:06:25.353410 29682 ts_manager.cc:194] Registered new tserver with Master: 1a90a8480fa14f169d39d639797882e0 (127.27.225.194:39377)
I20260806 21:06:25.354346 29682 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.194@KRBTEST.COM'} at 127.27.225.194:38629
WARNING: no policy specified for kudu/127.27.225.195@KRBTEST.COM; defaulting to no policy
I20260806 21:06:25.364293 29733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:25.353924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:38227 (local address 127.27.225.254:33913)
0806 21:06:25.354054 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:25.354059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:25.354338 (+ 279us) server_negotiation.cc:407] Connection header received
0806 21:06:25.354470 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:25.354473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:25.354518 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:25.354613 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:25.355555 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.356474 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.357919 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.358235 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.358353 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:25.360048 (+ 1695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:25.360055 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:25.360061 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:25.360099 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:25.362046 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:25.362645 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:25.362649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:25.362652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:25.362748 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:25.363160 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:25.363163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:25.363165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:25.363561 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:25.363664 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:25.363984 (+ 320us) server_negotiation.cc:299] Negotiation successful
0806 21:06:25.364105 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 21:06:25.364316 30043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:25.353887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:33913 (local address 127.27.225.194:38227)
0806 21:06:25.354230 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:25.354250 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:25.354369 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:25.354799 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:25.354805 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:25.355080 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:25.355375 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:25.355388 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.356856 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.356862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:25.357676 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:25.357688 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.358395 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.358403 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:25.358548 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:25.359281 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:25.359304 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:25.359813 (+ 509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:25.362239 (+ 2426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:25.362245 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:25.362249 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:25.362517 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:25.362879 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:25.362882 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:25.362884 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:25.363012 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:25.363688 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:25.363691 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:25.363814 (+ 123us) client_negotiation.cc:769] Sending connection context
0806 21:06:25.363920 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 21:06:25.364077 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":139,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
I20260806 21:06:25.386884 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:33913
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:44557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:25.519749 30045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:25.520098 30045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:25.520146 30045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:25.524514 30045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:25.524606 30045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:25.524649 30045 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 21:06:25.524684 30045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:06:25.530674 30045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:44557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.225.254:33913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:25.531836 30045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:25.532981 30045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:25.543816 30050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:25.543812 30053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:25.543812 30051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:25.544303 30045 server_base.cc:1034] running on GCE node
I20260806 21:06:25.544839 30045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:25.545586 30045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:25.546779 30045 hybrid_clock.cc:648] HybridClock initialized: now 1786050385546748 us; error 86 us; skew 500 ppm
Aug 06 21:06:25 dist-test-slave-69rg krb5kdc[29637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050385, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:25.550344 30045 init.cc:377] Logged in from keytab as kudu/127.27.225.195@KRBTEST.COM (short username kudu)
I20260806 21:06:25.551839 30045 webserver.cc:460] Webserver started at http://127.27.225.195:40089/ using document root <none> and password file <none>
I20260806 21:06:25.552672 30045 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:25.552798 30045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:25.553007 30045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:25.555603 30045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "40a33850ea1340e2b36881afb76fab8b"
format_stamp: "Formatted at 2026-08-06 21:06:25 on dist-test-slave-69rg"
server_key: "b03279c205a97f5d90117a013d3b7911"
server_key_iv: "e4903b6f2ff1626dc8685259f258dc2a"
server_key_version: "encryptionkey@0"
I20260806 21:06:25.556306 30045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "40a33850ea1340e2b36881afb76fab8b"
format_stamp: "Formatted at 2026-08-06 21:06:25 on dist-test-slave-69rg"
server_key: "b03279c205a97f5d90117a013d3b7911"
server_key_iv: "e4903b6f2ff1626dc8685259f258dc2a"
server_key_version: "encryptionkey@0"
I20260806 21:06:25.561276 30045 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 21:06:25.564244 30060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:25.565738 30045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:06:25.565940 30045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "40a33850ea1340e2b36881afb76fab8b"
format_stamp: "Formatted at 2026-08-06 21:06:25 on dist-test-slave-69rg"
server_key: "b03279c205a97f5d90117a013d3b7911"
server_key_iv: "e4903b6f2ff1626dc8685259f258dc2a"
server_key_version: "encryptionkey@0"
I20260806 21:06:25.566085 30045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:25.584570 30045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:25.588261 30045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:25.588490 30045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:25.590121 30045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:25.590197 30045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:25.590252 30045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:25.590281 30045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 21:06:25 dist-test-slave-69rg krb5kdc[29637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050385, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:25.603073 30045 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:40609
I20260806 21:06:25.603425 30178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:40609 every 8 connection(s)
I20260806 21:06:25.604235 30045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:06:25.606097 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 30045
I20260806 21:06:25.606238 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:06:25.606499 28551 external_mini_cluster.cc:1428] Setting key 9a1853e82f835577ba3b502b1711533b
I20260806 21:06:25.606601 29733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:25.591001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:52311 (local address 127.27.225.254:33913)
0806 21:06:25.591184 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:25.591191 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:25.592096 (+ 905us) server_negotiation.cc:407] Connection header received
0806 21:06:25.593208 (+ 1112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:25.593212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:25.593280 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:25.593369 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:25.595223 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.596365 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.598037 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.598488 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.598643 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:25.602031 (+ 3388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:25.602035 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:25.602038 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:25.602070 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:25.603980 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:25.604732 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:25.604736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:25.604738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:25.604844 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:25.605327 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:25.605330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:25.605332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:25.605560 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:25.605649 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:25.606288 (+ 639us) server_negotiation.cc:299] Negotiation successful
0806 21:06:25.606436 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 21:06:25.607668 30069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:25.591439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:33913 (local address 127.27.225.195:52311)
0806 21:06:25.591913 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:25.591965 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:25.592886 (+ 921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:25.593575 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:25.593587 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:25.594232 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:25.594997 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:25.595024 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.596575 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.596580 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:25.597882 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:25.597903 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.598708 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.598713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:25.598953 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:25.599644 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:25.599661 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:25.601748 (+ 2087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:25.604155 (+ 2407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:25.604161 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:25.604178 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:25.604578 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:25.604960 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:25.604965 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:25.604968 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:25.605163 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:25.605660 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:25.605665 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:25.606022 (+ 357us) client_negotiation.cc:769] Sending connection context
0806 21:06:25.606217 (+ 195us) client_negotiation.cc:240] Negotiation successful
0806 21:06:25.606609 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":148,"threads_started":1}
I20260806 21:06:25.609018 30179 heartbeater.cc:347] Connected to a master server at 127.27.225.254:33913
I20260806 21:06:25.609197 30179 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:25.609854 30179 heartbeater.cc:511] Master 127.27.225.254:33913 requested a full tablet report, sending...
I20260806 21:06:25.611213 29682 ts_manager.cc:194] Registered new tserver with Master: 40a33850ea1340e2b36881afb76fab8b (127.27.225.195:40609)
I20260806 21:06:25.612134 29682 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.195@KRBTEST.COM'} at 127.27.225.195:52311
I20260806 21:06:25.620064 29733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:25.611783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:37537 (local address 127.27.225.254:33913)
0806 21:06:25.611912 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:25.611918 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:25.612084 (+ 166us) server_negotiation.cc:407] Connection header received
0806 21:06:25.612181 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:25.612185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:25.612242 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:25.612337 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:25.612882 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.613743 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.614649 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.614885 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.614978 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:25.616261 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:25.616264 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:25.616266 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:25.616292 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:25.617929 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:25.618423 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:25.618426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:25.618428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:25.618530 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:25.618959 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:25.618963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:25.618964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:25.619269 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:25.619362 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:25.619733 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 21:06:25.619872 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 21:06:25.620101 30185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:25.611660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:33913 (local address 127.27.225.195:37537)
0806 21:06:25.611990 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:25.612010 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:25.612100 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:25.612377 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:25.612394 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:25.612652 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:25.612761 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:25.612768 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.613958 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.613961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:25.614517 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:25.614525 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.615023 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.615026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:25.615111 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:25.615750 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:25.615763 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:25.616108 (+ 345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:25.618060 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:25.618071 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:25.618073 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:25.618306 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:25.618688 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:25.618691 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:25.618692 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:25.618793 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:25.619395 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:25.619399 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:25.619528 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 21:06:25.619662 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 21:06:25.619877 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":127,"threads_started":1}
I20260806 21:06:25.621333 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 21:06:25 dist-test-slave-69rg krb5kdc[29637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050385, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 21:06:25 dist-test-slave-69rg krb5kdc[29637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050385, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:25.651103 29733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:25.641803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50784 (local address 127.27.225.254:33913)
0806 21:06:25.641983 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:25.641987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:25.641999 (+ 12us) server_negotiation.cc:407] Connection header received
0806 21:06:25.642041 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:25.642044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:25.642084 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:25.642177 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:25.643000 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.643547 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.644426 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.644651 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.644783 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:25.647573 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:25.647578 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:25.647580 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:25.647614 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:25.649172 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:25.649671 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:25.649674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:25.649676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:25.649775 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:25.650108 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:25.650111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:25.650113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:25.650322 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:25.650465 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:25.650771 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 21:06:25.650935 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 21:06:25.655054 29682 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50784:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:06:25.658496 29682 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:06:25.678524 30199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:25.670757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42058 (local address 127.27.225.195:40609)
0806 21:06:25.672234 (+ 1477us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:25.672247 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:25.672273 (+ 26us) server_negotiation.cc:407] Connection header received
0806 21:06:25.672357 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:25.672363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:25.672584 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:25.673195 (+ 611us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:25.673350 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.674581 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.676842 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.677635 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.677789 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:25.677956 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:25.678213 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 21:06:25.678352 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1368,"thread_start_us":122,"threads_started":1}
I20260806 21:06:25.678592 30197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:25.671301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:37399 (local address 127.0.0.1:60742)
0806 21:06:25.672129 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:25.672143 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:25.672220 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:25.672937 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:25.672940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:25.672968 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:25.673326 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:25.673335 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.674798 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.674801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:25.676341 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:25.676354 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.677585 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.677588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:25.677759 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:25.677816 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 21:06:25.678152 (+ 336us) client_negotiation.cc:240] Negotiation successful
0806 21:06:25.678405 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":750,"mutex_wait_us":371,"spinlock_wait_cycles":896,"thread_start_us":107,"threads_started":1}
I20260806 21:06:25.678598 30200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:25.670712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:40609 (local address 127.0.0.1:42058)
0806 21:06:25.671698 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:25.671764 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:25.671920 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:25.672773 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:25.672779 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:25.672802 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:25.672974 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:25.672981 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.674757 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.674760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:25.676681 (+ 1921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:25.676697 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.677798 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.677801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:25.678035 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:25.678075 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 21:06:25.678151 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 21:06:25.678405 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"mutex_wait_us":346,"thread_start_us":114,"threads_started":1}
I20260806 21:06:25.679306 30201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:25.671463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60742 (local address 127.27.225.193:37399)
0806 21:06:25.671872 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:25.671882 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:25.672330 (+ 448us) server_negotiation.cc:407] Connection header received
0806 21:06:25.672419 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:25.672424 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:25.672620 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:25.672769 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:25.673489 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.674658 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.676496 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.677319 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.677477 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:25.677653 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:25.678889 (+ 1236us) server_negotiation.cc:299] Negotiation successful
0806 21:06:25.679021 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":104,"threads_started":1}
I20260806 21:06:25.682036 29823 tablet_service.cc:1467] Processing CreateTablet for tablet fb44744e1039437eaa4e5035cabefd24 (DEFAULT_TABLE table=test-table [id=d6d8f3bb4f7d449fbbdeb772ad084c7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:25.682050 30110 tablet_service.cc:1467] Processing CreateTablet for tablet fb44744e1039437eaa4e5035cabefd24 (DEFAULT_TABLE table=test-table [id=d6d8f3bb4f7d449fbbdeb772ad084c7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:25.683126 29823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb44744e1039437eaa4e5035cabefd24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:25.683220 30110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb44744e1039437eaa4e5035cabefd24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:25.683912 30198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:25.670751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:39377 (local address 127.0.0.1:38510)
0806 21:06:25.671806 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:25.671821 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:25.671932 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:25.672907 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:25.672910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:25.672937 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:25.673165 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:25.673173 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.674633 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.674640 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:25.676681 (+ 2041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:25.676697 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.683351 (+ 6654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.683356 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:25.683495 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:25.683527 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 21:06:25.683631 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 21:06:25.683746 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":984,"mutex_wait_us":235,"spinlock_wait_cycles":2432,"thread_start_us":87,"threads_started":1}
I20260806 21:06:25.683949 30202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:25.671737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38510 (local address 127.27.225.194:39377)
0806 21:06:25.672275 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:25.672285 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:25.672306 (+ 21us) server_negotiation.cc:407] Connection header received
0806 21:06:25.672367 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:25.672371 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:25.672584 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:25.672751 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:25.673310 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.674332 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.677756 (+ 3424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.683064 (+ 5308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.683380 (+ 316us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:25.683543 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:25.683670 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 21:06:25.683791 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":106,"threads_started":1}
I20260806 21:06:25.687358 29967 tablet_service.cc:1467] Processing CreateTablet for tablet fb44744e1039437eaa4e5035cabefd24 (DEFAULT_TABLE table=test-table [id=d6d8f3bb4f7d449fbbdeb772ad084c7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:25.688578 29967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb44744e1039437eaa4e5035cabefd24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:25.692114 30203 tablet_bootstrap.cc:492] T fb44744e1039437eaa4e5035cabefd24 P b2c7539b602c4861a114b8912e36d1de: Bootstrap starting.
I20260806 21:06:25.692792 30204 tablet_bootstrap.cc:492] T fb44744e1039437eaa4e5035cabefd24 P 40a33850ea1340e2b36881afb76fab8b: Bootstrap starting.
I20260806 21:06:25.695330 30203 tablet_bootstrap.cc:654] T fb44744e1039437eaa4e5035cabefd24 P b2c7539b602c4861a114b8912e36d1de: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:25.695330 30204 tablet_bootstrap.cc:654] T fb44744e1039437eaa4e5035cabefd24 P 40a33850ea1340e2b36881afb76fab8b: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:25.696466 30203 log.cc:824] T fb44744e1039437eaa4e5035cabefd24 P b2c7539b602c4861a114b8912e36d1de: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:25.696493 30205 tablet_bootstrap.cc:492] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0: Bootstrap starting.
I20260806 21:06:25.696507 30204 log.cc:824] T fb44744e1039437eaa4e5035cabefd24 P 40a33850ea1340e2b36881afb76fab8b: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:25.698858 30204 tablet_bootstrap.cc:492] T fb44744e1039437eaa4e5035cabefd24 P 40a33850ea1340e2b36881afb76fab8b: No bootstrap required, opened a new log
I20260806 21:06:25.698858 30203 tablet_bootstrap.cc:492] T fb44744e1039437eaa4e5035cabefd24 P b2c7539b602c4861a114b8912e36d1de: No bootstrap required, opened a new log
I20260806 21:06:25.698944 30205 tablet_bootstrap.cc:654] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:25.699080 30203 ts_tablet_manager.cc:1397] T fb44744e1039437eaa4e5035cabefd24 P b2c7539b602c4861a114b8912e36d1de: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 21:06:25.699080 30204 ts_tablet_manager.cc:1397] T fb44744e1039437eaa4e5035cabefd24 P 40a33850ea1340e2b36881afb76fab8b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 21:06:25.699748 30205 log.cc:824] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:25.702308 30205 tablet_bootstrap.cc:492] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0: No bootstrap required, opened a new log
I20260806 21:06:25.702656 30205 ts_tablet_manager.cc:1397] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 21:06:25.703598 30204 raft_consensus.cc:348] T fb44744e1039437eaa4e5035cabefd24 P 40a33850ea1340e2b36881afb76fab8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } }
I20260806 21:06:25.703917 30204 raft_consensus.cc:374] T fb44744e1039437eaa4e5035cabefd24 P 40a33850ea1340e2b36881afb76fab8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:25.703986 30204 raft_consensus.cc:729] T fb44744e1039437eaa4e5035cabefd24 P 40a33850ea1340e2b36881afb76fab8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a33850ea1340e2b36881afb76fab8b, State: Initialized, Role: FOLLOWER
I20260806 21:06:25.703974 30203 raft_consensus.cc:348] T fb44744e1039437eaa4e5035cabefd24 P b2c7539b602c4861a114b8912e36d1de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } }
I20260806 21:06:25.704349 30203 raft_consensus.cc:374] T fb44744e1039437eaa4e5035cabefd24 P b2c7539b602c4861a114b8912e36d1de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:25.704413 30203 raft_consensus.cc:729] T fb44744e1039437eaa4e5035cabefd24 P b2c7539b602c4861a114b8912e36d1de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2c7539b602c4861a114b8912e36d1de, State: Initialized, Role: FOLLOWER
I20260806 21:06:25.704494 30204 consensus_queue.cc:260] T fb44744e1039437eaa4e5035cabefd24 P 40a33850ea1340e2b36881afb76fab8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } }
I20260806 21:06:25.704836 30203 consensus_queue.cc:260] T fb44744e1039437eaa4e5035cabefd24 P b2c7539b602c4861a114b8912e36d1de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } }
I20260806 21:06:25.705452 30179 heartbeater.cc:503] Master 127.27.225.254:33913 was elected leader, sending a full tablet report...
I20260806 21:06:25.705682 30204 ts_tablet_manager.cc:1428] T fb44744e1039437eaa4e5035cabefd24 P 40a33850ea1340e2b36881afb76fab8b: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 21:06:25.705682 30203 ts_tablet_manager.cc:1428] T fb44744e1039437eaa4e5035cabefd24 P b2c7539b602c4861a114b8912e36d1de: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 21:06:25.706079 29892 heartbeater.cc:503] Master 127.27.225.254:33913 was elected leader, sending a full tablet report...
I20260806 21:06:25.707535 30205 raft_consensus.cc:348] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } }
I20260806 21:06:25.707840 30205 raft_consensus.cc:374] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:25.707914 30205 raft_consensus.cc:729] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a90a8480fa14f169d39d639797882e0, State: Initialized, Role: FOLLOWER
I20260806 21:06:25.708377 30205 consensus_queue.cc:260] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } }
I20260806 21:06:25.709060 30036 heartbeater.cc:503] Master 127.27.225.254:33913 was elected leader, sending a full tablet report...
I20260806 21:06:25.709504 30205 ts_tablet_manager.cc:1428] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260806 21:06:25.711930 30211 raft_consensus.cc:482] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:06:25.712100 30211 raft_consensus.cc:504] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } }
I20260806 21:06:25.713251 30211 leader_election.cc:288] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40a33850ea1340e2b36881afb76fab8b (127.27.225.195:40609), b2c7539b602c4861a114b8912e36d1de (127.27.225.193:37399)
I20260806 21:06:25.717377 29926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:25.713459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:40609 (local address 127.27.225.194:40277)
0806 21:06:25.713579 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:25.713596 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:25.713679 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:25.713996 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:25.713999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:25.714018 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:25.714139 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:25.714146 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.715075 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.715078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:25.716114 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:25.716137 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.716937 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.716942 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:25.717048 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:25.717073 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 21:06:25.717155 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 21:06:25.717227 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 21:06:25.717407 30199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:25.713515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:40277 (local address 127.27.225.195:40609)
0806 21:06:25.713710 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:25.713716 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:25.713733 (+ 17us) server_negotiation.cc:407] Connection header received
0806 21:06:25.713790 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:25.713793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:25.713890 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:25.713974 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:25.714277 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.714978 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.716262 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.716825 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.716926 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:25.716969 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:25.717212 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 21:06:25.717301 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260806 21:06:25.718171 30132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb44744e1039437eaa4e5035cabefd24" candidate_uuid: "1a90a8480fa14f169d39d639797882e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a33850ea1340e2b36881afb76fab8b" is_pre_election: true
I20260806 21:06:25.718521 30132 raft_consensus.cc:2454] T fb44744e1039437eaa4e5035cabefd24 P 40a33850ea1340e2b36881afb76fab8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a90a8480fa14f169d39d639797882e0 in term 0.
I20260806 21:06:25.719017 30212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:25.713470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:37399 (local address 127.27.225.194:47185)
0806 21:06:25.714446 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:25.714458 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:25.714537 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:25.714760 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:25.714762 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:25.714774 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:25.714910 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:25.714914 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.715652 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.715655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:25.717540 (+ 1885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:25.717549 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.718623 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.718627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:25.718719 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:25.718742 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 21:06:25.718829 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 21:06:25.718898 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":922,"mutex_wait_us":581,"thread_start_us":139,"threads_started":1}
I20260806 21:06:25.719101 29918 leader_election.cc:302] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a90a8480fa14f169d39d639797882e0, 40a33850ea1340e2b36881afb76fab8b; no voters:
I20260806 21:06:25.719255 30201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:25.713569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:47185 (local address 127.27.225.193:37399)
0806 21:06:25.713696 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:25.713700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:25.714536 (+ 836us) server_negotiation.cc:407] Connection header received
0806 21:06:25.714612 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:25.714615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:25.714661 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:25.714737 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:25.715010 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.715539 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.717696 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.718360 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.718481 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:25.718533 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:25.719020 (+ 487us) server_negotiation.cc:299] Negotiation successful
0806 21:06:25.719120 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 21:06:25.719509 30211 raft_consensus.cc:2781] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:06:25.719610 30211 raft_consensus.cc:482] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:06:25.719671 30211 raft_consensus.cc:3037] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:25.719874 29844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb44744e1039437eaa4e5035cabefd24" candidate_uuid: "1a90a8480fa14f169d39d639797882e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c7539b602c4861a114b8912e36d1de" is_pre_election: true
I20260806 21:06:25.720187 29844 raft_consensus.cc:2454] T fb44744e1039437eaa4e5035cabefd24 P b2c7539b602c4861a114b8912e36d1de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a90a8480fa14f169d39d639797882e0 in term 0.
I20260806 21:06:25.721037 30211 raft_consensus.cc:504] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } }
I20260806 21:06:25.721374 30211 leader_election.cc:288] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0 [CANDIDATE]: Term 1 election: Requested vote from peers 40a33850ea1340e2b36881afb76fab8b (127.27.225.195:40609), b2c7539b602c4861a114b8912e36d1de (127.27.225.193:37399)
I20260806 21:06:25.721747 29844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb44744e1039437eaa4e5035cabefd24" candidate_uuid: "1a90a8480fa14f169d39d639797882e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c7539b602c4861a114b8912e36d1de"
I20260806 21:06:25.721750 30132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb44744e1039437eaa4e5035cabefd24" candidate_uuid: "1a90a8480fa14f169d39d639797882e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a33850ea1340e2b36881afb76fab8b"
I20260806 21:06:25.721902 30132 raft_consensus.cc:3037] T fb44744e1039437eaa4e5035cabefd24 P 40a33850ea1340e2b36881afb76fab8b [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:25.721903 29844 raft_consensus.cc:3037] T fb44744e1039437eaa4e5035cabefd24 P b2c7539b602c4861a114b8912e36d1de [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:25.723284 30132 raft_consensus.cc:2454] T fb44744e1039437eaa4e5035cabefd24 P 40a33850ea1340e2b36881afb76fab8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a90a8480fa14f169d39d639797882e0 in term 1.
I20260806 21:06:25.723290 29844 raft_consensus.cc:2454] T fb44744e1039437eaa4e5035cabefd24 P b2c7539b602c4861a114b8912e36d1de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a90a8480fa14f169d39d639797882e0 in term 1.
I20260806 21:06:25.723682 29921 leader_election.cc:302] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a90a8480fa14f169d39d639797882e0, b2c7539b602c4861a114b8912e36d1de; no voters:
I20260806 21:06:25.723894 30211 raft_consensus.cc:2781] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:25.724143 30211 raft_consensus.cc:686] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0 [term 1 LEADER]: Becoming Leader. State: Replica: 1a90a8480fa14f169d39d639797882e0, State: Running, Role: LEADER
I20260806 21:06:25.724530 30211 consensus_queue.cc:237] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } }
I20260806 21:06:25.728508 29681 catalog_manager.cc:5495] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a90a8480fa14f169d39d639797882e0 (127.27.225.194). New cstate: current_term: 1 leader_uuid: "1a90a8480fa14f169d39d639797882e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } health_report { overall_health: UNKNOWN } } }
I20260806 21:06:25.758842 30202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:25.755014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38512 (local address 127.27.225.194:39377)
0806 21:06:25.755144 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:25.755147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:25.755164 (+ 17us) server_negotiation.cc:407] Connection header received
0806 21:06:25.755325 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:25.755328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:25.755388 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:25.755486 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:06:25.755767 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.756268 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.757101 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:25.757324 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:25.757400 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:25.757780 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:06:25.758393 (+ 613us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:06:25.758505 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:25.758655 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 21:06:25.758713 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 21:06:25.766114 29844 raft_consensus.cc:1261] T fb44744e1039437eaa4e5035cabefd24 P b2c7539b602c4861a114b8912e36d1de [term 1 FOLLOWER]: Refusing update from remote peer 1a90a8480fa14f169d39d639797882e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:06:25.766232 30132 raft_consensus.cc:1261] T fb44744e1039437eaa4e5035cabefd24 P 40a33850ea1340e2b36881afb76fab8b [term 1 FOLLOWER]: Refusing update from remote peer 1a90a8480fa14f169d39d639797882e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:06:25.767062 30213 consensus_queue.cc:1035] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:06:25.768838 30213 consensus_queue.cc:1035] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:06:25.777187 30218 mvcc.cc:205] Tried to move back new op lower bound from 7315662380087746560 to 7315662379930877952. Current Snapshot: MvccSnapshot[applied={T|T < 7315662380087746560}]
I20260806 21:06:25.792400 29681 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:06:25.792706 29681 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:06:25.795696 29681 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2366e370d4a040c7b31d453c341abe4c: Sending DeleteTablet for 3 replicas of tablet fb44744e1039437eaa4e5035cabefd24
I20260806 21:06:25.796733 30110 tablet_service.cc:1514] Processing DeleteTablet for tablet fb44744e1039437eaa4e5035cabefd24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:25 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:42058
I20260806 21:06:25.797149 29967 tablet_service.cc:1514] Processing DeleteTablet for tablet fb44744e1039437eaa4e5035cabefd24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:25 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:38510
I20260806 21:06:25.797523 30227 tablet_replica.cc:330] T fb44744e1039437eaa4e5035cabefd24 P 40a33850ea1340e2b36881afb76fab8b: stopping tablet replica
I20260806 21:06:25.797474 29823 tablet_service.cc:1514] Processing DeleteTablet for tablet fb44744e1039437eaa4e5035cabefd24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:25 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:60742
I20260806 21:06:25.797988 30228 tablet_replica.cc:330] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0: stopping tablet replica
I20260806 21:06:25.798118 30227 raft_consensus.cc:2229] T fb44744e1039437eaa4e5035cabefd24 P 40a33850ea1340e2b36881afb76fab8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:06:25.798322 30228 raft_consensus.cc:2229] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:06:25.798422 29681 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50784:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 21:06:25.798707 30228 raft_consensus.cc:2258] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:25.798820 30227 raft_consensus.cc:2258] T fb44744e1039437eaa4e5035cabefd24 P 40a33850ea1340e2b36881afb76fab8b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 21:06:25.798936 29681 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:06:25.799111 30229 tablet_replica.cc:330] T fb44744e1039437eaa4e5035cabefd24 P b2c7539b602c4861a114b8912e36d1de: stopping tablet replica
I20260806 21:06:25.799553 30229 raft_consensus.cc:2229] T fb44744e1039437eaa4e5035cabefd24 P b2c7539b602c4861a114b8912e36d1de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:06:25.799865 30229 raft_consensus.cc:2258] T fb44744e1039437eaa4e5035cabefd24 P b2c7539b602c4861a114b8912e36d1de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:25.801436 30229 ts_tablet_manager.cc:1905] T fb44744e1039437eaa4e5035cabefd24 P b2c7539b602c4861a114b8912e36d1de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:25.804663 30228 ts_tablet_manager.cc:1905] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:25.805260 30110 tablet_service.cc:1467] Processing CreateTablet for tablet 3cd0fb4b7083485d89857fddf4036557 (DEFAULT_TABLE table=test-table [id=482652ec74aa4f98903450cb1f2ca5ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:25.805683 30110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cd0fb4b7083485d89857fddf4036557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:25.805693 29967 tablet_service.cc:1467] Processing CreateTablet for tablet 3cd0fb4b7083485d89857fddf4036557 (DEFAULT_TABLE table=test-table [id=482652ec74aa4f98903450cb1f2ca5ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:25.806054 29967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cd0fb4b7083485d89857fddf4036557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:25.806187 29823 tablet_service.cc:1467] Processing CreateTablet for tablet 3cd0fb4b7083485d89857fddf4036557 (DEFAULT_TABLE table=test-table [id=482652ec74aa4f98903450cb1f2ca5ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:25.808194 30228 ts_tablet_manager.cc:1918] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:06:25.808336 30228 log.cc:1196] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-1/wal/wals/fb44744e1039437eaa4e5035cabefd24
I20260806 21:06:25.808754 30228 ts_tablet_manager.cc:1939] T fb44744e1039437eaa4e5035cabefd24 P 1a90a8480fa14f169d39d639797882e0: Deleting consensus metadata
I20260806 21:06:25.808838 30205 tablet_bootstrap.cc:492] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0: Bootstrap starting.
I20260806 21:06:25.809218 30204 tablet_bootstrap.cc:492] T 3cd0fb4b7083485d89857fddf4036557 P 40a33850ea1340e2b36881afb76fab8b: Bootstrap starting.
I20260806 21:06:25.810074 30205 tablet_bootstrap.cc:654] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:25.810174 29669 catalog_manager.cc:4841] TS 1a90a8480fa14f169d39d639797882e0 (127.27.225.194:39377): tablet fb44744e1039437eaa4e5035cabefd24 (table test-table [id=d6d8f3bb4f7d449fbbdeb772ad084c7f]) successfully deleted
I20260806 21:06:25.810384 30227 ts_tablet_manager.cc:1905] T fb44744e1039437eaa4e5035cabefd24 P 40a33850ea1340e2b36881afb76fab8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:25.810461 30204 tablet_bootstrap.cc:654] T 3cd0fb4b7083485d89857fddf4036557 P 40a33850ea1340e2b36881afb76fab8b: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:25.811659 30205 tablet_bootstrap.cc:492] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0: No bootstrap required, opened a new log
I20260806 21:06:25.811774 30205 ts_tablet_manager.cc:1397] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:06:25.811779 29823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cd0fb4b7083485d89857fddf4036557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:25.812345 30205 raft_consensus.cc:348] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } }
I20260806 21:06:25.812477 30205 raft_consensus.cc:374] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:25.812510 30205 raft_consensus.cc:729] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a90a8480fa14f169d39d639797882e0, State: Initialized, Role: FOLLOWER
I20260806 21:06:25.812794 30205 consensus_queue.cc:260] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } }
I20260806 21:06:25.813298 30205 ts_tablet_manager.cc:1428] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 21:06:25.815291 30203 tablet_bootstrap.cc:492] T 3cd0fb4b7083485d89857fddf4036557 P b2c7539b602c4861a114b8912e36d1de: Bootstrap starting.
I20260806 21:06:25.816771 30227 ts_tablet_manager.cc:1918] T fb44744e1039437eaa4e5035cabefd24 P 40a33850ea1340e2b36881afb76fab8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:06:25.816864 30203 tablet_bootstrap.cc:654] T 3cd0fb4b7083485d89857fddf4036557 P b2c7539b602c4861a114b8912e36d1de: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:25.817023 30227 log.cc:1196] T fb44744e1039437eaa4e5035cabefd24 P 40a33850ea1340e2b36881afb76fab8b: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/fb44744e1039437eaa4e5035cabefd24
I20260806 21:06:25.817432 30227 ts_tablet_manager.cc:1939] T fb44744e1039437eaa4e5035cabefd24 P 40a33850ea1340e2b36881afb76fab8b: Deleting consensus metadata
I20260806 21:06:25.817924 30229 ts_tablet_manager.cc:1918] T fb44744e1039437eaa4e5035cabefd24 P b2c7539b602c4861a114b8912e36d1de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:06:25.818056 30229 log.cc:1196] T fb44744e1039437eaa4e5035cabefd24 P b2c7539b602c4861a114b8912e36d1de: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-0/wal/wals/fb44744e1039437eaa4e5035cabefd24
I20260806 21:06:25.818768 30204 tablet_bootstrap.cc:492] T 3cd0fb4b7083485d89857fddf4036557 P 40a33850ea1340e2b36881afb76fab8b: No bootstrap required, opened a new log
I20260806 21:06:25.818879 30204 ts_tablet_manager.cc:1397] T 3cd0fb4b7083485d89857fddf4036557 P 40a33850ea1340e2b36881afb76fab8b: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260806 21:06:25.819077 30203 tablet_bootstrap.cc:492] T 3cd0fb4b7083485d89857fddf4036557 P b2c7539b602c4861a114b8912e36d1de: No bootstrap required, opened a new log
I20260806 21:06:25.819172 30203 ts_tablet_manager.cc:1397] T 3cd0fb4b7083485d89857fddf4036557 P b2c7539b602c4861a114b8912e36d1de: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 21:06:25.819499 30204 raft_consensus.cc:348] T 3cd0fb4b7083485d89857fddf4036557 P 40a33850ea1340e2b36881afb76fab8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } }
I20260806 21:06:25.819669 30204 raft_consensus.cc:374] T 3cd0fb4b7083485d89857fddf4036557 P 40a33850ea1340e2b36881afb76fab8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:25.819710 30204 raft_consensus.cc:729] T 3cd0fb4b7083485d89857fddf4036557 P 40a33850ea1340e2b36881afb76fab8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a33850ea1340e2b36881afb76fab8b, State: Initialized, Role: FOLLOWER
I20260806 21:06:25.819904 30204 consensus_queue.cc:260] T 3cd0fb4b7083485d89857fddf4036557 P 40a33850ea1340e2b36881afb76fab8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } }
I20260806 21:06:25.819991 30203 raft_consensus.cc:348] T 3cd0fb4b7083485d89857fddf4036557 P b2c7539b602c4861a114b8912e36d1de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } }
I20260806 21:06:25.820142 30203 raft_consensus.cc:374] T 3cd0fb4b7083485d89857fddf4036557 P b2c7539b602c4861a114b8912e36d1de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:25.820194 30203 raft_consensus.cc:729] T 3cd0fb4b7083485d89857fddf4036557 P b2c7539b602c4861a114b8912e36d1de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2c7539b602c4861a114b8912e36d1de, State: Initialized, Role: FOLLOWER
I20260806 21:06:25.820354 30203 consensus_queue.cc:260] T 3cd0fb4b7083485d89857fddf4036557 P b2c7539b602c4861a114b8912e36d1de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } }
I20260806 21:06:25.820812 30203 ts_tablet_manager.cc:1428] T 3cd0fb4b7083485d89857fddf4036557 P b2c7539b602c4861a114b8912e36d1de: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 21:06:25.821416 30204 ts_tablet_manager.cc:1428] T 3cd0fb4b7083485d89857fddf4036557 P 40a33850ea1340e2b36881afb76fab8b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 21:06:25.822001 30229 ts_tablet_manager.cc:1939] T fb44744e1039437eaa4e5035cabefd24 P b2c7539b602c4861a114b8912e36d1de: Deleting consensus metadata
I20260806 21:06:25.822847 29667 catalog_manager.cc:4841] TS 40a33850ea1340e2b36881afb76fab8b (127.27.225.195:40609): tablet fb44744e1039437eaa4e5035cabefd24 (table test-table [id=d6d8f3bb4f7d449fbbdeb772ad084c7f]) successfully deleted
I20260806 21:06:25.823712 29670 catalog_manager.cc:4841] TS b2c7539b602c4861a114b8912e36d1de (127.27.225.193:37399): tablet fb44744e1039437eaa4e5035cabefd24 (table test-table [id=d6d8f3bb4f7d449fbbdeb772ad084c7f]) successfully deleted
W20260806 21:06:25.824697 29893 tablet.cc:2369] T 3cd0fb4b7083485d89857fddf4036557 P b2c7539b602c4861a114b8912e36d1de: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 21:06:25.847883 30037 tablet.cc:2369] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 21:06:25.855412 30180 tablet.cc:2369] T 3cd0fb4b7083485d89857fddf4036557 P 40a33850ea1340e2b36881afb76fab8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:06:25.965159 30221 raft_consensus.cc:482] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:06:25.965353 30221 raft_consensus.cc:504] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } }
I20260806 21:06:25.965942 30221 leader_election.cc:288] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2c7539b602c4861a114b8912e36d1de (127.27.225.193:37399), 40a33850ea1340e2b36881afb76fab8b (127.27.225.195:40609)
I20260806 21:06:25.966300 29844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cd0fb4b7083485d89857fddf4036557" candidate_uuid: "1a90a8480fa14f169d39d639797882e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c7539b602c4861a114b8912e36d1de" is_pre_election: true
I20260806 21:06:25.966413 30132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cd0fb4b7083485d89857fddf4036557" candidate_uuid: "1a90a8480fa14f169d39d639797882e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a33850ea1340e2b36881afb76fab8b" is_pre_election: true
I20260806 21:06:25.966475 29844 raft_consensus.cc:2454] T 3cd0fb4b7083485d89857fddf4036557 P b2c7539b602c4861a114b8912e36d1de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a90a8480fa14f169d39d639797882e0 in term 0.
I20260806 21:06:25.966557 30132 raft_consensus.cc:2454] T 3cd0fb4b7083485d89857fddf4036557 P 40a33850ea1340e2b36881afb76fab8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a90a8480fa14f169d39d639797882e0 in term 0.
I20260806 21:06:25.966789 29921 leader_election.cc:302] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a90a8480fa14f169d39d639797882e0, b2c7539b602c4861a114b8912e36d1de; no voters:
I20260806 21:06:25.966980 30221 raft_consensus.cc:2781] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:06:25.967051 30221 raft_consensus.cc:482] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:06:25.967092 30221 raft_consensus.cc:3037] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:25.968137 30221 raft_consensus.cc:504] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } }
I20260806 21:06:25.968554 30221 leader_election.cc:288] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0 [CANDIDATE]: Term 1 election: Requested vote from peers b2c7539b602c4861a114b8912e36d1de (127.27.225.193:37399), 40a33850ea1340e2b36881afb76fab8b (127.27.225.195:40609)
I20260806 21:06:25.968945 29844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cd0fb4b7083485d89857fddf4036557" candidate_uuid: "1a90a8480fa14f169d39d639797882e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c7539b602c4861a114b8912e36d1de"
I20260806 21:06:25.969103 29844 raft_consensus.cc:3037] T 3cd0fb4b7083485d89857fddf4036557 P b2c7539b602c4861a114b8912e36d1de [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:25.969039 30132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cd0fb4b7083485d89857fddf4036557" candidate_uuid: "1a90a8480fa14f169d39d639797882e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a33850ea1340e2b36881afb76fab8b"
I20260806 21:06:25.969192 30132 raft_consensus.cc:3037] T 3cd0fb4b7083485d89857fddf4036557 P 40a33850ea1340e2b36881afb76fab8b [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:25.970294 29844 raft_consensus.cc:2454] T 3cd0fb4b7083485d89857fddf4036557 P b2c7539b602c4861a114b8912e36d1de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a90a8480fa14f169d39d639797882e0 in term 1.
I20260806 21:06:25.970305 30132 raft_consensus.cc:2454] T 3cd0fb4b7083485d89857fddf4036557 P 40a33850ea1340e2b36881afb76fab8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a90a8480fa14f169d39d639797882e0 in term 1.
I20260806 21:06:25.970667 29921 leader_election.cc:302] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a90a8480fa14f169d39d639797882e0, b2c7539b602c4861a114b8912e36d1de; no voters:
I20260806 21:06:25.970861 30221 raft_consensus.cc:2781] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:25.970975 30221 raft_consensus.cc:686] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0 [term 1 LEADER]: Becoming Leader. State: Replica: 1a90a8480fa14f169d39d639797882e0, State: Running, Role: LEADER
I20260806 21:06:25.971151 30221 consensus_queue.cc:237] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } }
I20260806 21:06:25.973176 29683 catalog_manager.cc:5495] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a90a8480fa14f169d39d639797882e0 (127.27.225.194). New cstate: current_term: 1 leader_uuid: "1a90a8480fa14f169d39d639797882e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } health_report { overall_health: UNKNOWN } } }
Aug 06 21:06:26 dist-test-slave-69rg krb5kdc[29637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050384, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:26.041695 29733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:26.031959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50796 (local address 127.27.225.254:33913)
0806 21:06:26.032172 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:26.032178 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:26.032300 (+ 122us) server_negotiation.cc:407] Connection header received
0806 21:06:26.032481 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:26.032484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:26.032519 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:26.032603 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:26.033389 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:26.034031 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:26.034858 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:26.035080 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:26.035196 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:26.037896 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:26.037900 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:26.037902 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:26.037928 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:26.039622 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:26.040142 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:26.040147 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:26.040149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:26.040301 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:26.040616 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:26.040621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:26.040623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:26.040868 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:26.040989 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:26.041352 (+ 363us) server_negotiation.cc:299] Negotiation successful
0806 21:06:26.041500 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 21:06:26.041723 30239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:26.031856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:33913 (local address 127.0.0.1:50796)
0806 21:06:26.032199 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:26.032213 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:26.032361 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:26.032706 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:26.032709 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:26.033153 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:26.033259 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:26.033266 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:26.034145 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:26.034152 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:26.034733 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:26.034741 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:26.035205 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:26.035208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:26.035290 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:26.035958 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:26.035972 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:26.037677 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:26.039741 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:26.039757 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:26.039765 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:26.040028 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:26.040418 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:26.040421 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:26.040423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:26.040510 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:26.041007 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:26.041015 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:26.041180 (+ 165us) client_negotiation.cc:769] Sending connection context
0806 21:06:26.041306 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 21:06:26.041575 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":89,"thread_start_us":110,"threads_started":1}
I20260806 21:06:26.047019 29683 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:50796:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 21:06:26.048051 29683 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:06:26.054394 30110 tablet_service.cc:1467] Processing CreateTablet for tablet b4aa2fcc84564082919a23c416c4f035 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c51c9158d14b43af939da1789fd0ae2f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 21:06:26.054533 29823 tablet_service.cc:1467] Processing CreateTablet for tablet b4aa2fcc84564082919a23c416c4f035 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c51c9158d14b43af939da1789fd0ae2f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 21:06:26.054934 30110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4aa2fcc84564082919a23c416c4f035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:26.054957 29823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4aa2fcc84564082919a23c416c4f035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:26.058130 30203 tablet_bootstrap.cc:492] T b4aa2fcc84564082919a23c416c4f035 P b2c7539b602c4861a114b8912e36d1de: Bootstrap starting.
I20260806 21:06:26.058604 29967 tablet_service.cc:1467] Processing CreateTablet for tablet b4aa2fcc84564082919a23c416c4f035 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c51c9158d14b43af939da1789fd0ae2f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 21:06:26.059267 30203 tablet_bootstrap.cc:654] T b4aa2fcc84564082919a23c416c4f035 P b2c7539b602c4861a114b8912e36d1de: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:26.059731 29967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4aa2fcc84564082919a23c416c4f035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:26.060863 30203 tablet_bootstrap.cc:492] T b4aa2fcc84564082919a23c416c4f035 P b2c7539b602c4861a114b8912e36d1de: No bootstrap required, opened a new log
I20260806 21:06:26.060971 30203 ts_tablet_manager.cc:1397] T b4aa2fcc84564082919a23c416c4f035 P b2c7539b602c4861a114b8912e36d1de: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:06:26.061264 30204 tablet_bootstrap.cc:492] T b4aa2fcc84564082919a23c416c4f035 P 40a33850ea1340e2b36881afb76fab8b: Bootstrap starting.
I20260806 21:06:26.061547 30203 raft_consensus.cc:348] T b4aa2fcc84564082919a23c416c4f035 P b2c7539b602c4861a114b8912e36d1de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } }
I20260806 21:06:26.061692 30203 raft_consensus.cc:374] T b4aa2fcc84564082919a23c416c4f035 P b2c7539b602c4861a114b8912e36d1de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:26.061748 30203 raft_consensus.cc:729] T b4aa2fcc84564082919a23c416c4f035 P b2c7539b602c4861a114b8912e36d1de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2c7539b602c4861a114b8912e36d1de, State: Initialized, Role: FOLLOWER
I20260806 21:06:26.061935 30203 consensus_queue.cc:260] T b4aa2fcc84564082919a23c416c4f035 P b2c7539b602c4861a114b8912e36d1de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } }
I20260806 21:06:26.062321 30203 ts_tablet_manager.cc:1428] T b4aa2fcc84564082919a23c416c4f035 P b2c7539b602c4861a114b8912e36d1de: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 21:06:26.062477 30204 tablet_bootstrap.cc:654] T b4aa2fcc84564082919a23c416c4f035 P 40a33850ea1340e2b36881afb76fab8b: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:26.063249 30210 tablet_replica.cc:439] T b4aa2fcc84564082919a23c416c4f035 P b2c7539b602c4861a114b8912e36d1de: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } } }
I20260806 21:06:26.063433 30210 tablet_replica.cc:442] T b4aa2fcc84564082919a23c416c4f035 P b2c7539b602c4861a114b8912e36d1de: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:06:26.064940 30204 tablet_bootstrap.cc:492] T b4aa2fcc84564082919a23c416c4f035 P 40a33850ea1340e2b36881afb76fab8b: No bootstrap required, opened a new log
I20260806 21:06:26.065056 30204 ts_tablet_manager.cc:1397] T b4aa2fcc84564082919a23c416c4f035 P 40a33850ea1340e2b36881afb76fab8b: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260806 21:06:26.065578 30204 raft_consensus.cc:348] T b4aa2fcc84564082919a23c416c4f035 P 40a33850ea1340e2b36881afb76fab8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } }
I20260806 21:06:26.065713 30204 raft_consensus.cc:374] T b4aa2fcc84564082919a23c416c4f035 P 40a33850ea1340e2b36881afb76fab8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:26.065750 30204 raft_consensus.cc:729] T b4aa2fcc84564082919a23c416c4f035 P 40a33850ea1340e2b36881afb76fab8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a33850ea1340e2b36881afb76fab8b, State: Initialized, Role: FOLLOWER
I20260806 21:06:26.065919 30204 consensus_queue.cc:260] T b4aa2fcc84564082919a23c416c4f035 P 40a33850ea1340e2b36881afb76fab8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } }
I20260806 21:06:26.066226 30204 ts_tablet_manager.cc:1428] T b4aa2fcc84564082919a23c416c4f035 P 40a33850ea1340e2b36881afb76fab8b: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260806 21:06:26.066298 30209 tablet_replica.cc:439] T b4aa2fcc84564082919a23c416c4f035 P 40a33850ea1340e2b36881afb76fab8b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } } }
I20260806 21:06:26.066464 30209 tablet_replica.cc:442] T b4aa2fcc84564082919a23c416c4f035 P 40a33850ea1340e2b36881afb76fab8b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:06:26.067888 30205 tablet_bootstrap.cc:492] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0: Bootstrap starting.
I20260806 21:06:26.068696 30205 tablet_bootstrap.cc:654] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:26.070158 30205 tablet_bootstrap.cc:492] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0: No bootstrap required, opened a new log
I20260806 21:06:26.070281 30205 ts_tablet_manager.cc:1397] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 21:06:26.070770 30205 raft_consensus.cc:348] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } }
I20260806 21:06:26.070925 30205 raft_consensus.cc:374] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:26.070972 30205 raft_consensus.cc:729] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a90a8480fa14f169d39d639797882e0, State: Initialized, Role: FOLLOWER
I20260806 21:06:26.071112 30205 consensus_queue.cc:260] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } }
I20260806 21:06:26.071425 30205 ts_tablet_manager.cc:1428] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 21:06:26.071401 30221 tablet_replica.cc:439] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } } }
I20260806 21:06:26.071570 30221 tablet_replica.cc:442] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:06:26.077432 30221 raft_consensus.cc:482] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:06:26.077580 30221 raft_consensus.cc:504] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } }
I20260806 21:06:26.078084 30221 leader_election.cc:288] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2c7539b602c4861a114b8912e36d1de (127.27.225.193:37399), 40a33850ea1340e2b36881afb76fab8b (127.27.225.195:40609)
I20260806 21:06:26.078490 29844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4aa2fcc84564082919a23c416c4f035" candidate_uuid: "1a90a8480fa14f169d39d639797882e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c7539b602c4861a114b8912e36d1de" is_pre_election: true
I20260806 21:06:26.078639 29844 raft_consensus.cc:2454] T b4aa2fcc84564082919a23c416c4f035 P b2c7539b602c4861a114b8912e36d1de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a90a8480fa14f169d39d639797882e0 in term 0.
I20260806 21:06:26.078593 30132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4aa2fcc84564082919a23c416c4f035" candidate_uuid: "1a90a8480fa14f169d39d639797882e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a33850ea1340e2b36881afb76fab8b" is_pre_election: true
I20260806 21:06:26.078769 30132 raft_consensus.cc:2454] T b4aa2fcc84564082919a23c416c4f035 P 40a33850ea1340e2b36881afb76fab8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a90a8480fa14f169d39d639797882e0 in term 0.
I20260806 21:06:26.078969 29921 leader_election.cc:302] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a90a8480fa14f169d39d639797882e0, b2c7539b602c4861a114b8912e36d1de; no voters:
I20260806 21:06:26.079226 30221 raft_consensus.cc:2781] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:06:26.079290 30221 raft_consensus.cc:482] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:06:26.079335 30221 raft_consensus.cc:3037] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:26.080456 30221 raft_consensus.cc:504] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } }
I20260806 21:06:26.080822 30221 leader_election.cc:288] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0 [CANDIDATE]: Term 1 election: Requested vote from peers b2c7539b602c4861a114b8912e36d1de (127.27.225.193:37399), 40a33850ea1340e2b36881afb76fab8b (127.27.225.195:40609)
I20260806 21:06:26.081141 29844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4aa2fcc84564082919a23c416c4f035" candidate_uuid: "1a90a8480fa14f169d39d639797882e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c7539b602c4861a114b8912e36d1de"
I20260806 21:06:26.081189 30132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4aa2fcc84564082919a23c416c4f035" candidate_uuid: "1a90a8480fa14f169d39d639797882e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40a33850ea1340e2b36881afb76fab8b"
I20260806 21:06:26.081287 29844 raft_consensus.cc:3037] T b4aa2fcc84564082919a23c416c4f035 P b2c7539b602c4861a114b8912e36d1de [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:26.081338 30132 raft_consensus.cc:3037] T b4aa2fcc84564082919a23c416c4f035 P 40a33850ea1340e2b36881afb76fab8b [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:26.082360 29844 raft_consensus.cc:2454] T b4aa2fcc84564082919a23c416c4f035 P b2c7539b602c4861a114b8912e36d1de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a90a8480fa14f169d39d639797882e0 in term 1.
I20260806 21:06:26.082571 30132 raft_consensus.cc:2454] T b4aa2fcc84564082919a23c416c4f035 P 40a33850ea1340e2b36881afb76fab8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a90a8480fa14f169d39d639797882e0 in term 1.
I20260806 21:06:26.082685 29921 leader_election.cc:302] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a90a8480fa14f169d39d639797882e0, b2c7539b602c4861a114b8912e36d1de; no voters:
I20260806 21:06:26.082886 30221 raft_consensus.cc:2781] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:26.082999 30221 raft_consensus.cc:686] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0 [term 1 LEADER]: Becoming Leader. State: Replica: 1a90a8480fa14f169d39d639797882e0, State: Running, Role: LEADER
I20260806 21:06:26.083148 30221 consensus_queue.cc:237] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } }
I20260806 21:06:26.083839 30213 tablet_replica.cc:439] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0: TxnStatusTablet state changed. Reason: New leader 1a90a8480fa14f169d39d639797882e0. Latest consensus state: current_term: 1 leader_uuid: "1a90a8480fa14f169d39d639797882e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } } }
I20260806 21:06:26.083942 30213 tablet_replica.cc:442] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0: This TxnStatusTablet replica's current role is: LEADER
I20260806 21:06:26.084890 29683 catalog_manager.cc:5495] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a90a8480fa14f169d39d639797882e0 (127.27.225.194). New cstate: current_term: 1 leader_uuid: "1a90a8480fa14f169d39d639797882e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } health_report { overall_health: UNKNOWN } } }
I20260806 21:06:26.085784 30244 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 21:06:26.085992 30244 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 21:06:26.101859 30202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:26.097941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38524 (local address 127.27.225.194:39377)
0806 21:06:26.098116 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:26.098121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:26.098140 (+ 19us) server_negotiation.cc:407] Connection header received
0806 21:06:26.098214 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:26.098217 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:26.098281 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:26.098363 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:06:26.098633 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:26.099456 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:26.100519 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:26.100851 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:26.100960 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:26.101314 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:06:26.101396 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:06:26.101476 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:26.101603 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 21:06:26.101660 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 21:06:26.101861 30239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:26.097735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:39377 (local address 127.0.0.1:38524)
0806 21:06:26.097957 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:26.097977 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:26.098112 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:26.098407 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:26.098409 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:26.098417 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 21:06:26.098520 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:26.098524 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:26.099650 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:26.099655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:26.100390 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:26.100403 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:26.100963 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:26.100966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:26.101072 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:26.101132 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 21:06:26.101481 (+ 349us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 21:06:26.101504 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 21:06:26.101588 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 21:06:26.101691 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 21:06:26.105273 29844 raft_consensus.cc:1261] T b4aa2fcc84564082919a23c416c4f035 P b2c7539b602c4861a114b8912e36d1de [term 1 FOLLOWER]: Refusing update from remote peer 1a90a8480fa14f169d39d639797882e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:06:26.105347 30132 raft_consensus.cc:1261] T b4aa2fcc84564082919a23c416c4f035 P 40a33850ea1340e2b36881afb76fab8b [term 1 FOLLOWER]: Refusing update from remote peer 1a90a8480fa14f169d39d639797882e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:06:26.105870 30221 consensus_queue.cc:1035] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:06:26.106053 30213 consensus_queue.cc:1035] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:06:26.107758 30210 tablet_replica.cc:439] T b4aa2fcc84564082919a23c416c4f035 P b2c7539b602c4861a114b8912e36d1de: TxnStatusTablet state changed. Reason: New leader 1a90a8480fa14f169d39d639797882e0. Latest consensus state: current_term: 1 leader_uuid: "1a90a8480fa14f169d39d639797882e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } } }
I20260806 21:06:26.107923 30210 tablet_replica.cc:442] T b4aa2fcc84564082919a23c416c4f035 P b2c7539b602c4861a114b8912e36d1de: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:06:26.108791 30209 tablet_replica.cc:439] T b4aa2fcc84564082919a23c416c4f035 P 40a33850ea1340e2b36881afb76fab8b: TxnStatusTablet state changed. Reason: New leader 1a90a8480fa14f169d39d639797882e0. Latest consensus state: current_term: 1 leader_uuid: "1a90a8480fa14f169d39d639797882e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } } }
I20260806 21:06:26.108932 30209 tablet_replica.cc:442] T b4aa2fcc84564082919a23c416c4f035 P 40a33850ea1340e2b36881afb76fab8b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:06:26.110641 30213 tablet_replica.cc:439] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1a90a8480fa14f169d39d639797882e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } } }
I20260806 21:06:26.110783 30213 tablet_replica.cc:442] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0: This TxnStatusTablet replica's current role is: LEADER
I20260806 21:06:26.111382 30210 tablet_replica.cc:439] T b4aa2fcc84564082919a23c416c4f035 P b2c7539b602c4861a114b8912e36d1de: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1a90a8480fa14f169d39d639797882e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } } }
I20260806 21:06:26.111505 30210 tablet_replica.cc:442] T b4aa2fcc84564082919a23c416c4f035 P b2c7539b602c4861a114b8912e36d1de: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:06:26.111707 30217 mvcc.cc:205] Tried to move back new op lower bound from 7315662381482311680 to 7315662381398028288. Current Snapshot: MvccSnapshot[applied={T|T < 7315662381482311680}]
I20260806 21:06:26.111963 30216 mvcc.cc:205] Tried to move back new op lower bound from 7315662381482311680 to 7315662381398028288. Current Snapshot: MvccSnapshot[applied={T|T < 7315662381482311680 or (T in {7315662381482311680})}]
I20260806 21:06:26.112756 30211 tablet_replica.cc:439] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1a90a8480fa14f169d39d639797882e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } } }
I20260806 21:06:26.112870 30211 tablet_replica.cc:442] T b4aa2fcc84564082919a23c416c4f035 P 1a90a8480fa14f169d39d639797882e0: This TxnStatusTablet replica's current role is: LEADER
I20260806 21:06:26.115212 30209 tablet_replica.cc:439] T b4aa2fcc84564082919a23c416c4f035 P 40a33850ea1340e2b36881afb76fab8b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1a90a8480fa14f169d39d639797882e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 } } peers { permanent_uuid: "1a90a8480fa14f169d39d639797882e0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39377 } } peers { permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 } } }
I20260806 21:06:26.115348 30209 tablet_replica.cc:442] T b4aa2fcc84564082919a23c416c4f035 P 40a33850ea1340e2b36881afb76fab8b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:06:26.129508 30202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:26.125045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:37465 (local address 127.27.225.194:39377)
0806 21:06:26.125197 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:26.125203 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:26.125461 (+ 258us) server_negotiation.cc:407] Connection header received
0806 21:06:26.125577 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:26.125579 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:26.125736 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:26.125871 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:06:26.126187 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:26.126809 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:26.128180 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:26.128506 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:26.128603 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:26.129004 (+ 401us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:06:26.129096 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:06:26.129166 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:26.129295 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 21:06:26.129346 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":63}
I20260806 21:06:26.129565 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:26.124990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:39377 (local address 127.27.225.194:37465)
0806 21:06:26.125340 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:26.125356 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:26.125488 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:26.125873 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:26.125876 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:26.125885 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 21:06:26.126027 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:26.126033 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:26.126970 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:26.126974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:26.127963 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:26.127976 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:26.128652 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:26.128655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:26.128747 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:26.128825 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 21:06:26.129197 (+ 372us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 21:06:26.129204 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 21:06:26.129275 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 21:06:26.129375 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":19,"thread_start_us":131,"threads_started":1}
I20260806 21:06:26.138967 29844 raft_consensus.cc:1261] T 3cd0fb4b7083485d89857fddf4036557 P b2c7539b602c4861a114b8912e36d1de [term 1 FOLLOWER]: Refusing update from remote peer 1a90a8480fa14f169d39d639797882e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:06:26.139035 30132 raft_consensus.cc:1261] T 3cd0fb4b7083485d89857fddf4036557 P 40a33850ea1340e2b36881afb76fab8b [term 1 FOLLOWER]: Refusing update from remote peer 1a90a8480fa14f169d39d639797882e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:06:26.139472 30213 consensus_queue.cc:1035] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2c7539b602c4861a114b8912e36d1de" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:06:26.139802 30213 consensus_queue.cc:1035] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40a33850ea1340e2b36881afb76fab8b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:06:26.191156 29683 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:06:26.191345 29683 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:06:26.194013 29683 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2366e370d4a040c7b31d453c341abe4c: Sending DeleteTablet for 3 replicas of tablet 3cd0fb4b7083485d89857fddf4036557
I20260806 21:06:26.194885 29823 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cd0fb4b7083485d89857fddf4036557 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:26 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:60742
I20260806 21:06:26.194988 30110 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cd0fb4b7083485d89857fddf4036557 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:26 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:42058
I20260806 21:06:26.195114 30229 tablet_replica.cc:330] T 3cd0fb4b7083485d89857fddf4036557 P b2c7539b602c4861a114b8912e36d1de: stopping tablet replica
I20260806 21:06:26.195190 30227 tablet_replica.cc:330] T 3cd0fb4b7083485d89857fddf4036557 P 40a33850ea1340e2b36881afb76fab8b: stopping tablet replica
I20260806 21:06:26.195320 30227 raft_consensus.cc:2229] T 3cd0fb4b7083485d89857fddf4036557 P 40a33850ea1340e2b36881afb76fab8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:06:26.195314 30229 raft_consensus.cc:2229] T 3cd0fb4b7083485d89857fddf4036557 P b2c7539b602c4861a114b8912e36d1de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:06:26.195449 30227 raft_consensus.cc:2258] T 3cd0fb4b7083485d89857fddf4036557 P 40a33850ea1340e2b36881afb76fab8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:26.195489 30229 raft_consensus.cc:2258] T 3cd0fb4b7083485d89857fddf4036557 P b2c7539b602c4861a114b8912e36d1de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:26.196635 30229 ts_tablet_manager.cc:1905] T 3cd0fb4b7083485d89857fddf4036557 P b2c7539b602c4861a114b8912e36d1de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:26.196977 30227 ts_tablet_manager.cc:1905] T 3cd0fb4b7083485d89857fddf4036557 P 40a33850ea1340e2b36881afb76fab8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:26.200721 30229 ts_tablet_manager.cc:1918] T 3cd0fb4b7083485d89857fddf4036557 P b2c7539b602c4861a114b8912e36d1de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:06:26.200852 30229 log.cc:1196] T 3cd0fb4b7083485d89857fddf4036557 P b2c7539b602c4861a114b8912e36d1de: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-0/wal/wals/3cd0fb4b7083485d89857fddf4036557
I20260806 21:06:26.200973 29967 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cd0fb4b7083485d89857fddf4036557 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:26 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:38510
I20260806 21:06:26.201169 30229 ts_tablet_manager.cc:1939] T 3cd0fb4b7083485d89857fddf4036557 P b2c7539b602c4861a114b8912e36d1de: Deleting consensus metadata
I20260806 21:06:26.201220 30228 tablet_replica.cc:330] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0: stopping tablet replica
I20260806 21:06:26.201359 30228 raft_consensus.cc:2229] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:06:26.201596 30228 raft_consensus.cc:2258] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:26.202402 29670 catalog_manager.cc:4841] TS b2c7539b602c4861a114b8912e36d1de (127.27.225.193:37399): tablet 3cd0fb4b7083485d89857fddf4036557 (table test-table [id=482652ec74aa4f98903450cb1f2ca5ec]) successfully deleted
I20260806 21:06:26.202792 30228 ts_tablet_manager.cc:1905] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:26.206182 30227 ts_tablet_manager.cc:1918] T 3cd0fb4b7083485d89857fddf4036557 P 40a33850ea1340e2b36881afb76fab8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:06:26.206364 30227 log.cc:1196] T 3cd0fb4b7083485d89857fddf4036557 P 40a33850ea1340e2b36881afb76fab8b: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/3cd0fb4b7083485d89857fddf4036557
I20260806 21:06:26.206754 30227 ts_tablet_manager.cc:1939] T 3cd0fb4b7083485d89857fddf4036557 P 40a33850ea1340e2b36881afb76fab8b: Deleting consensus metadata
I20260806 21:06:26.207020 30228 ts_tablet_manager.cc:1918] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:06:26.207123 30228 log.cc:1196] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050378084738-28551-0/minicluster-data/ts-1/wal/wals/3cd0fb4b7083485d89857fddf4036557
I20260806 21:06:26.207679 30228 ts_tablet_manager.cc:1939] T 3cd0fb4b7083485d89857fddf4036557 P 1a90a8480fa14f169d39d639797882e0: Deleting consensus metadata
I20260806 21:06:26.207906 29667 catalog_manager.cc:4841] TS 40a33850ea1340e2b36881afb76fab8b (127.27.225.195:40609): tablet 3cd0fb4b7083485d89857fddf4036557 (table test-table [id=482652ec74aa4f98903450cb1f2ca5ec]) successfully deleted
Aug 06 21:06:26 dist-test-slave-69rg krb5kdc[29637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050385, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.225.193@KRBTEST.COM
I20260806 21:06:26.209121 29669 catalog_manager.cc:4841] TS 1a90a8480fa14f169d39d639797882e0 (127.27.225.194:39377): tablet 3cd0fb4b7083485d89857fddf4036557 (table test-table [id=482652ec74aa4f98903450cb1f2ca5ec]) successfully deleted
I20260806 21:06:26.214784 30201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:26.196675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60748 (local address 127.27.225.193:37399)
0806 21:06:26.196809 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:26.196814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:26.197953 (+ 1139us) server_negotiation.cc:407] Connection header received
0806 21:06:26.198289 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:26.198294 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:26.198357 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:26.198422 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:26.199664 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:26.200538 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:26.202660 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:26.203089 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:26.203206 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:26.209272 (+ 6066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:26.209293 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:26.209388 (+ 95us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:26.209433 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:26.211973 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:26.212927 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:26.212935 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:26.212937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:26.213059 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:26.213413 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:26.213416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:26.213418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:26.213758 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:26.214077 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:26.214318 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 21:06:26.214473 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 21:06:26.215387 29886 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60748
I20260806 21:06:26.228348 29733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:26.217440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50804 (local address 127.27.225.254:33913)
0806 21:06:26.217584 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:26.217590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:26.217696 (+ 106us) server_negotiation.cc:407] Connection header received
0806 21:06:26.217965 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:26.217967 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:26.218019 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:26.218106 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:26.219185 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:26.219906 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:26.221405 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:26.221677 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:26.221791 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:26.223982 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:26.223985 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:26.223987 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:26.224028 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:26.225992 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:26.226606 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:26.226618 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:26.226619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:26.226734 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:26.227115 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:26.227119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:26.227120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:26.227358 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:26.227481 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:26.227954 (+ 473us) server_negotiation.cc:299] Negotiation successful
0806 21:06:26.228081 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260806 21:06:26.228912 29683 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50804
I20260806 21:06:26.230557 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 29757
I20260806 21:06:26.240409 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 29901
I20260806 21:06:26.249204 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 30045
I20260806 21:06:26.256834 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 29651
2026-08-06T21:06:26Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2923 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:06:26 dist-test-slave-69rg krb5kdc[30263](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:06:26 dist-test-slave-69rg krb5kdc[30263](info): set up 2 sockets
Aug 06 21:06:26 dist-test-slave-69rg krb5kdc[30263](info): commencing operation
krb5kdc: starting...
W20260806 21:06:27.439864 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:06:27 dist-test-slave-69rg krb5kdc[30263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:06:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:06:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:06:27.611667 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:38963
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:42007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:38963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:27.742195 30279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:27.742625 30279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:27.742744 30279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:27.748046 30279 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:06:27.748190 30279 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 21:06:27.748222 30279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:27.748265 30279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:27.748310 30279 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:06:27.748351 30279 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:06:27.754945 30279 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:42007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:38963
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:38963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:27.756248 30279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:27.757417 30279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:27.767374 30285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:27.767390 30284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:27.767374 30287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:27.767671 30279 server_base.cc:1034] running on GCE node
I20260806 21:06:27.768558 30279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:27.769912 30279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:27.771144 30279 hybrid_clock.cc:648] HybridClock initialized: now 1786050387771121 us; error 39 us; skew 500 ppm
Aug 06 21:06:27 dist-test-slave-69rg krb5kdc[30263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050387, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:27.774513 30279 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:06:27.775668 30279 webserver.cc:460] Webserver started at http://127.27.225.254:40109/ using document root <none> and password file <none>
I20260806 21:06:27.776322 30279 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:27.776396 30279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:27.776652 30279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:27.779592 30279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "2508f2d546dd43549606b08b33b1c4de"
format_stamp: "Formatted at 2026-08-06 21:06:27 on dist-test-slave-69rg"
server_key: "57774a1156dfc6980a8c787a931d7199"
server_key_iv: "0b913099dee65a95ff1693f582c3d4a3"
server_key_version: "encryptionkey@0"
I20260806 21:06:27.780330 30279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "2508f2d546dd43549606b08b33b1c4de"
format_stamp: "Formatted at 2026-08-06 21:06:27 on dist-test-slave-69rg"
server_key: "57774a1156dfc6980a8c787a931d7199"
server_key_iv: "0b913099dee65a95ff1693f582c3d4a3"
server_key_version: "encryptionkey@0"
I20260806 21:06:27.784827 30279 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 21:06:27.788022 30294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:27.790642 30279 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 21:06:27.790855 30279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "2508f2d546dd43549606b08b33b1c4de"
format_stamp: "Formatted at 2026-08-06 21:06:27 on dist-test-slave-69rg"
server_key: "57774a1156dfc6980a8c787a931d7199"
server_key_iv: "0b913099dee65a95ff1693f582c3d4a3"
server_key_version: "encryptionkey@0"
I20260806 21:06:27.791023 30279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:27.810401 30279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:27.814646 30279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:27.814927 30279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:27.824644 30356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:38963 every 8 connection(s)
I20260806 21:06:27.824715 30279 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:38963
I20260806 21:06:27.825973 30279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:06:27.829135 30357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:27.831681 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 30279
I20260806 21:06:27.831840 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:06:27.832116 28551 external_mini_cluster.cc:1428] Setting key 7d5d603b7cf5ecb220a65250b9375bb3
I20260806 21:06:27.835733 30357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de: Bootstrap starting.
I20260806 21:06:27.838581 30357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:27.839506 30357 log.cc:824] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de: Log is configured to *not* fsync() on all Append() calls
Aug 06 21:06:27 dist-test-slave-69rg krb5kdc[30263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:27.842298 30357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de: No bootstrap required, opened a new log
I20260806 21:06:27.847345 30357 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2508f2d546dd43549606b08b33b1c4de" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 38963 } }
I20260806 21:06:27.847719 30357 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:27.847780 30357 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2508f2d546dd43549606b08b33b1c4de, State: Initialized, Role: FOLLOWER
I20260806 21:06:27.848280 30357 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2508f2d546dd43549606b08b33b1c4de" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 38963 } }
I20260806 21:06:27.848448 30357 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:06:27.848538 30357 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:06:27.848639 30357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:27.849242 30360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:27.833682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42676 (local address 127.27.225.254:38963)
0806 21:06:27.834175 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:27.834192 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:27.834229 (+ 37us) server_negotiation.cc:407] Connection header received
0806 21:06:27.835017 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:27.835040 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:27.835408 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:27.835734 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:27.836761 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:27.837893 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:27.839120 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:27.839652 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:27.839853 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:27.843084 (+ 3231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:27.843093 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:27.843117 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:27.843163 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:27.845469 (+ 2306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:27.846087 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:27.846093 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:27.846099 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:27.846271 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:27.846663 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:27.846666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:27.846668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:27.847240 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:27.847450 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:27.847727 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 21:06:27.848061 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":192,"threads_started":1}
I20260806 21:06:27.849933 30357 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2508f2d546dd43549606b08b33b1c4de" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 38963 } }
I20260806 21:06:27.850296 30357 leader_election.cc:302] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2508f2d546dd43549606b08b33b1c4de; no voters:
I20260806 21:06:27.850687 30357 leader_election.cc:288] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:06:27.850814 30362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:27.851042 30362 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de [term 1 LEADER]: Becoming Leader. State: Replica: 2508f2d546dd43549606b08b33b1c4de, State: Running, Role: LEADER
I20260806 21:06:27.851403 30362 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2508f2d546dd43549606b08b33b1c4de" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 38963 } }
I20260806 21:06:27.852089 30357 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:06:27.852672 30362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2508f2d546dd43549606b08b33b1c4de. Latest consensus state: current_term: 1 leader_uuid: "2508f2d546dd43549606b08b33b1c4de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2508f2d546dd43549606b08b33b1c4de" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 38963 } } }
I20260806 21:06:27.852798 30362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:27.853207 30368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:06:27.853309 30363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2508f2d546dd43549606b08b33b1c4de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2508f2d546dd43549606b08b33b1c4de" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 38963 } } }
I20260806 21:06:27.853487 30363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:27.856184 30368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:06:27.862233 30368 catalog_manager.cc:1347] Generated new cluster ID: 3bb9e5de40054688a05a8db045377b6d
I20260806 21:06:27.862344 30368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:06:27.887141 30368 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:06:27.888166 30368 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:06:27.896224 30368 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de: Generated new TSK 0
I20260806 21:06:27.897042 30368 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.225.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
I20260806 21:06:27.933465 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:38963
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:42007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:28.049861 30383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:28.050145 30383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:28.050202 30383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:28.054518 30383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:28.054586 30383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:28.054629 30383 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 21:06:28.054644 30383 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 21:06:28.054679 30383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:06:28.061437 30383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:42007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.225.254:38963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:28.062987 30383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:28.064414 30383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:06:28.078778 30383 server_base.cc:1034] running on GCE node
W20260806 21:06:28.078647 30389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:28.078647 30388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:28.078651 30391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:28.079532 30383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:28.080320 30383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:28.081489 30383 hybrid_clock.cc:648] HybridClock initialized: now 1786050388081440 us; error 66 us; skew 500 ppm
Aug 06 21:06:28 dist-test-slave-69rg krb5kdc[30263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050388, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:28.084538 30383 init.cc:377] Logged in from keytab as kudu/127.27.225.193@KRBTEST.COM (short username kudu)
I20260806 21:06:28.085474 30383 webserver.cc:460] Webserver started at http://127.27.225.193:34507/ using document root <none> and password file <none>
I20260806 21:06:28.086198 30383 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:28.086316 30383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:28.086601 30383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:28.088800 30383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "4c660a9fb7174be199d18350f472d637"
format_stamp: "Formatted at 2026-08-06 21:06:28 on dist-test-slave-69rg"
server_key: "73292b2a0037b364dbd1a173985f5967"
server_key_iv: "22b1a6be50360a19ff4b4de54d37d91e"
server_key_version: "encryptionkey@0"
I20260806 21:06:28.089341 30383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "4c660a9fb7174be199d18350f472d637"
format_stamp: "Formatted at 2026-08-06 21:06:28 on dist-test-slave-69rg"
server_key: "73292b2a0037b364dbd1a173985f5967"
server_key_iv: "22b1a6be50360a19ff4b4de54d37d91e"
server_key_version: "encryptionkey@0"
I20260806 21:06:28.093263 30383 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:06:28.095948 30398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:28.097266 30383 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:06:28.097397 30383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "4c660a9fb7174be199d18350f472d637"
format_stamp: "Formatted at 2026-08-06 21:06:28 on dist-test-slave-69rg"
server_key: "73292b2a0037b364dbd1a173985f5967"
server_key_iv: "22b1a6be50360a19ff4b4de54d37d91e"
server_key_version: "encryptionkey@0"
I20260806 21:06:28.097539 30383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:28.121991 30383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:28.125542 30383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:28.125764 30383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:28.127388 30383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:28.127466 30383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:28.127534 30383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:28.127578 30383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 21:06:28 dist-test-slave-69rg krb5kdc[30263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050388, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:28.139109 30383 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:40909
I20260806 21:06:28.139185 30516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:40909 every 8 connection(s)
I20260806 21:06:28.140060 30383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:06:28.141746 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 30383
I20260806 21:06:28.141916 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:06:28.142220 28551 external_mini_cluster.cc:1428] Setting key 590301002a1d994ef1fb8b59b275734d
I20260806 21:06:28.142684 30360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.128368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:38077 (local address 127.27.225.254:38963)
0806 21:06:28.128542 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:28.128547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:28.129491 (+ 944us) server_negotiation.cc:407] Connection header received
0806 21:06:28.130637 (+ 1146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:28.130641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:28.130713 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:28.130816 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:28.132201 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.133037 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.134533 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.134834 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.134979 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.138107 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:28.138111 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:28.138113 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:28.138143 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:28.140149 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:28.140891 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:28.140897 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:28.140902 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:28.141015 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:28.141405 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:28.141408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:28.141410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:28.141623 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:28.141740 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:28.142359 (+ 619us) server_negotiation.cc:299] Negotiation successful
0806 21:06:28.142502 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 21:06:28.143630 30409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.128811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:38963 (local address 127.27.225.193:38077)
0806 21:06:28.129297 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:28.129352 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:28.130380 (+ 1028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:28.130992 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:28.130999 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:28.131400 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:28.132037 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.132054 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.133203 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.133207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.134382 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.134394 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.135081 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.135086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.135365 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.136170 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:28.136187 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:28.137875 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:28.140309 (+ 2434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:28.140315 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:28.140331 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:28.140723 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:28.141128 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:28.141133 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:28.141136 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:28.141295 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:28.141756 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:28.141764 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:28.142114 (+ 350us) client_negotiation.cc:769] Sending connection context
0806 21:06:28.142304 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 21:06:28.142642 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":147,"threads_started":1}
I20260806 21:06:28.147081 30517 heartbeater.cc:347] Connected to a master server at 127.27.225.254:38963
I20260806 21:06:28.147272 30517 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:28.147840 30517 heartbeater.cc:511] Master 127.27.225.254:38963 requested a full tablet report, sending...
I20260806 21:06:28.149670 30311 ts_manager.cc:194] Registered new tserver with Master: 4c660a9fb7174be199d18350f472d637 (127.27.225.193:40909)
I20260806 21:06:28.151108 30311 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.193@KRBTEST.COM'} at 127.27.225.193:38077
WARNING: no policy specified for kudu/127.27.225.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.194@KRBTEST.COM" created.
I20260806 21:06:28.159101 30360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.149765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:34051 (local address 127.27.225.254:38963)
0806 21:06:28.149945 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:28.149952 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:28.150266 (+ 314us) server_negotiation.cc:407] Connection header received
0806 21:06:28.150426 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:28.150429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:28.150479 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:28.150561 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:28.151251 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.152005 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.153191 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.153755 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.154034 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.155342 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:28.155346 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:28.155348 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:28.155373 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:28.157014 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:28.157481 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:28.157514 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:28.157516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:28.157685 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:28.158098 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:28.158105 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:28.158156 (+ 51us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:28.158405 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:28.158494 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:28.158833 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 21:06:28.158957 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 21:06:28.159291 30525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.149641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:38963 (local address 127.27.225.193:34051)
0806 21:06:28.150147 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:28.150165 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:28.150297 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:28.150619 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:28.150624 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:28.150915 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:28.151093 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.151105 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.152153 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.152157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.153044 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.153056 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.154023 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.154027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.154116 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.154670 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:28.154683 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:28.155026 (+ 343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:28.157135 (+ 2109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:28.157138 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:28.157139 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:28.157356 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:28.157795 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:28.157821 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:28.157826 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:28.157996 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:28.158513 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:28.158516 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:28.158642 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 21:06:28.158765 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 21:06:28.158956 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":5,"thread_start_us":103,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
I20260806 21:06:28.178548 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:38963
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:42007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:28.296696 30527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:28.296983 30527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:28.297058 30527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:28.301435 30527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:28.301508 30527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:28.301581 30527 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 21:06:28.301623 30527 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 21:06:28.301687 30527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:06:28.307375 30527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:42007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.225.254:38963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:28.308542 30527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:28.309598 30527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:28.319653 30532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:28.319789 30527 server_base.cc:1034] running on GCE node
W20260806 21:06:28.319659 30535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:28.319659 30533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:28.320497 30527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:28.321214 30527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:28.322428 30527 hybrid_clock.cc:648] HybridClock initialized: now 1786050388322399 us; error 48 us; skew 500 ppm
Aug 06 21:06:28 dist-test-slave-69rg krb5kdc[30263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050388, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:28.330089 30527 init.cc:377] Logged in from keytab as kudu/127.27.225.194@KRBTEST.COM (short username kudu)
I20260806 21:06:28.331070 30527 webserver.cc:460] Webserver started at http://127.27.225.194:39617/ using document root <none> and password file <none>
I20260806 21:06:28.331637 30527 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:28.331708 30527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:28.331904 30527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:28.334146 30527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "754e07fa5de64d52abb908bfa66dc766"
format_stamp: "Formatted at 2026-08-06 21:06:28 on dist-test-slave-69rg"
server_key: "7dcfaee1ecc7ec33edb803d900da1547"
server_key_iv: "3df08f81a5b3e91d1f1b25a2239e120e"
server_key_version: "encryptionkey@0"
I20260806 21:06:28.334707 30527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "754e07fa5de64d52abb908bfa66dc766"
format_stamp: "Formatted at 2026-08-06 21:06:28 on dist-test-slave-69rg"
server_key: "7dcfaee1ecc7ec33edb803d900da1547"
server_key_iv: "3df08f81a5b3e91d1f1b25a2239e120e"
server_key_version: "encryptionkey@0"
I20260806 21:06:28.338830 30527 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 21:06:28.341568 30543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:28.343036 30527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 21:06:28.343189 30527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "754e07fa5de64d52abb908bfa66dc766"
format_stamp: "Formatted at 2026-08-06 21:06:28 on dist-test-slave-69rg"
server_key: "7dcfaee1ecc7ec33edb803d900da1547"
server_key_iv: "3df08f81a5b3e91d1f1b25a2239e120e"
server_key_version: "encryptionkey@0"
I20260806 21:06:28.343319 30527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:28.383172 30527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:28.386264 30527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:28.386490 30527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:28.387953 30527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:28.388007 30527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:28.388073 30527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:28.388106 30527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 21:06:28 dist-test-slave-69rg krb5kdc[30263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050388, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:28.399763 30527 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:39567
I20260806 21:06:28.400009 30660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:39567 every 8 connection(s)
I20260806 21:06:28.400864 30527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:06:28.404376 30360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.388975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:47289 (local address 127.27.225.254:38963)
0806 21:06:28.389125 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:28.389129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:28.390204 (+ 1075us) server_negotiation.cc:407] Connection header received
0806 21:06:28.391237 (+ 1033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:28.391242 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:28.391302 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:28.391433 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:28.393209 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.394151 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.396249 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.396495 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.396621 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.400142 (+ 3521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:28.400150 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:28.400156 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:28.400200 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:28.402185 (+ 1985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:28.402759 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:28.402762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:28.402764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:28.402865 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:28.403228 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:28.403231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:28.403232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:28.403449 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:28.403536 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:28.404085 (+ 549us) server_negotiation.cc:299] Negotiation successful
0806 21:06:28.404219 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 21:06:28.405232 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.389514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:38963 (local address 127.27.225.194:47289)
0806 21:06:28.390053 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:28.390096 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:28.390974 (+ 878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:28.391659 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:28.391671 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:28.392220 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:28.392959 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.392988 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.394328 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.394332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.396026 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.396040 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.396685 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.396689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.396919 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.397705 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:28.397732 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:28.399876 (+ 2144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:28.402306 (+ 2430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:28.402311 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:28.402322 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:28.402593 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:28.402974 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:28.402976 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:28.402978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:28.403118 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:28.403548 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:28.403554 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:28.403815 (+ 261us) client_negotiation.cc:769] Sending connection context
0806 21:06:28.403984 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 21:06:28.404264 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":144,"threads_started":1}
I20260806 21:06:28.406620 30661 heartbeater.cc:347] Connected to a master server at 127.27.225.254:38963
I20260806 21:06:28.407059 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 30527
I20260806 21:06:28.407182 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:06:28.407524 28551 external_mini_cluster.cc:1428] Setting key 57e584cbc6edc619c79229f32af03f6d
I20260806 21:06:28.408033 30661 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:28.408663 30661 heartbeater.cc:511] Master 127.27.225.254:38963 requested a full tablet report, sending...
I20260806 21:06:28.409735 30310 ts_manager.cc:194] Registered new tserver with Master: 754e07fa5de64d52abb908bfa66dc766 (127.27.225.194:39567)
I20260806 21:06:28.410532 30310 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.194@KRBTEST.COM'} at 127.27.225.194:47289
I20260806 21:06:28.419806 30667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.409251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:38963 (local address 127.27.225.194:44533)
0806 21:06:28.409800 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:28.409849 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:28.409990 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:28.410303 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:28.410308 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:28.410578 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:28.410768 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.410779 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.412054 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.412059 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.413161 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.413179 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.413920 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.413924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.414014 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.414648 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:28.414661 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:28.415047 (+ 386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:28.417973 (+ 2926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:28.417977 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:28.417979 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:28.418252 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:28.418624 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:28.418627 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:28.418628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:28.418734 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:28.419298 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:28.419300 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:28.419392 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 21:06:28.419479 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 21:06:28.419613 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":138,"threads_started":1}
I20260806 21:06:28.419975 30360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.409312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:44533 (local address 127.27.225.254:38963)
0806 21:06:28.409451 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:28.409455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:28.409917 (+ 462us) server_negotiation.cc:407] Connection header received
0806 21:06:28.410110 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:28.410114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:28.410160 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:28.410249 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:28.411142 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.411813 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.413379 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.413702 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.413862 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.415226 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:28.415233 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:28.415239 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:28.415287 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:28.417762 (+ 2475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:28.418385 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:28.418391 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:28.418393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:28.418508 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:28.418834 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:28.418838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:28.418840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:28.419197 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:28.419325 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:28.419583 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 21:06:28.419739 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":52}
WARNING: no policy specified for kudu/127.27.225.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
I20260806 21:06:28.442286 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:38963
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:42007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:28.562107 30671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:28.562431 30671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:28.562484 30671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:28.566658 30671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:28.566740 30671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:28.566784 30671 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 21:06:28.566799 30671 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 21:06:28.566833 30671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:06:28.575357 30671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:42007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.225.254:38963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:28.577111 30671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:28.578855 30671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:28.591349 30679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:28.591362 30676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:28.591349 30677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:28.591935 30671 server_base.cc:1034] running on GCE node
I20260806 21:06:28.592427 30671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:28.593148 30671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:28.594360 30671 hybrid_clock.cc:648] HybridClock initialized: now 1786050388594340 us; error 41 us; skew 500 ppm
Aug 06 21:06:28 dist-test-slave-69rg krb5kdc[30263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050388, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:28.597587 30671 init.cc:377] Logged in from keytab as kudu/127.27.225.195@KRBTEST.COM (short username kudu)
I20260806 21:06:28.598665 30671 webserver.cc:460] Webserver started at http://127.27.225.195:44557/ using document root <none> and password file <none>
I20260806 21:06:28.599300 30671 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:28.599368 30671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:28.599608 30671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:28.602154 30671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "a85bbe87cbd64819965452624929cdf5"
format_stamp: "Formatted at 2026-08-06 21:06:28 on dist-test-slave-69rg"
server_key: "fe9d9e91dfa31fe9bd667887af434ad4"
server_key_iv: "67279681432b991a93bb761c9a5123ce"
server_key_version: "encryptionkey@0"
I20260806 21:06:28.602771 30671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "a85bbe87cbd64819965452624929cdf5"
format_stamp: "Formatted at 2026-08-06 21:06:28 on dist-test-slave-69rg"
server_key: "fe9d9e91dfa31fe9bd667887af434ad4"
server_key_iv: "67279681432b991a93bb761c9a5123ce"
server_key_version: "encryptionkey@0"
I20260806 21:06:28.607131 30671 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 21:06:28.610069 30686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:28.611523 30671 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:06:28.611687 30671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "a85bbe87cbd64819965452624929cdf5"
format_stamp: "Formatted at 2026-08-06 21:06:28 on dist-test-slave-69rg"
server_key: "fe9d9e91dfa31fe9bd667887af434ad4"
server_key_iv: "67279681432b991a93bb761c9a5123ce"
server_key_version: "encryptionkey@0"
I20260806 21:06:28.611829 30671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:28.637634 30671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:28.641219 30671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:28.641448 30671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:28.643117 30671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:28.643172 30671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:28.643210 30671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:28.643265 30671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:28.655459 30671 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:42839
Aug 06 21:06:28 dist-test-slave-69rg krb5kdc[30263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050388, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:28.656462 30671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:06:28.658782 30803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:42839 every 8 connection(s)
I20260806 21:06:28.661125 30360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.644495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:41417 (local address 127.27.225.254:38963)
0806 21:06:28.644652 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:28.644659 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:28.645803 (+ 1144us) server_negotiation.cc:407] Connection header received
0806 21:06:28.647488 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:28.647492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:28.647575 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:28.647682 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:28.649581 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.650638 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.652807 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.653298 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.653393 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.657273 (+ 3880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:28.657279 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:28.657284 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:28.657321 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:28.659014 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:28.659577 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:28.659581 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:28.659583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:28.659672 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:28.660016 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:28.660020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:28.660022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:28.660214 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:28.660324 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:28.660818 (+ 494us) server_negotiation.cc:299] Negotiation successful
0806 21:06:28.660950 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 21:06:28.661178 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 30671
I20260806 21:06:28.661317 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:06:28.661688 28551 external_mini_cluster.cc:1428] Setting key d4b7b4bbf58935c3974c52ad856960fe
I20260806 21:06:28.661978 30695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.644996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:38963 (local address 127.27.225.195:41417)
0806 21:06:28.645580 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:28.645645 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:28.647130 (+ 1485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:28.648054 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:28.648065 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:28.648627 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:28.649371 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.649394 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.650904 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.650909 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.652627 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.652643 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.653682 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.653687 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.653974 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.654691 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:28.654716 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:28.657026 (+ 2310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:28.659171 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:28.659177 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:28.659192 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:28.659465 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:28.659768 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:28.659772 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:28.659774 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:28.659916 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:28.660325 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:28.660333 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:28.660575 (+ 242us) client_negotiation.cc:769] Sending connection context
0806 21:06:28.660762 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 21:06:28.661083 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":144,"threads_started":1}
I20260806 21:06:28.663234 30804 heartbeater.cc:347] Connected to a master server at 127.27.225.254:38963
I20260806 21:06:28.663409 30804 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:28.663928 30804 heartbeater.cc:511] Master 127.27.225.254:38963 requested a full tablet report, sending...
I20260806 21:06:28.665020 30311 ts_manager.cc:194] Registered new tserver with Master: a85bbe87cbd64819965452624929cdf5 (127.27.225.195:42839)
I20260806 21:06:28.665622 30311 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.195@KRBTEST.COM'} at 127.27.225.195:41417
I20260806 21:06:28.667600 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:06:28.675573 30360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.666271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:43197 (local address 127.27.225.254:38963)
0806 21:06:28.666380 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:28.666385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:28.666885 (+ 500us) server_negotiation.cc:407] Connection header received
0806 21:06:28.667083 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:28.667101 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:28.667174 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:28.667233 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:28.668042 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.668814 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.670298 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.670686 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.670820 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.672314 (+ 1494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:28.672317 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:28.672319 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:28.672344 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:28.673873 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:28.674339 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:28.674342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:28.674344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:28.674429 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:28.674717 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:28.674720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:28.674722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:28.674912 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:28.675021 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:28.675280 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 21:06:28.675392 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 21:06:28.675581 30811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.666223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:38963 (local address 127.27.225.195:43197)
0806 21:06:28.666791 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:28.666809 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:28.666966 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:28.667402 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:28.667406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:28.667682 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:28.667877 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.667885 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.668974 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.668982 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.670166 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.670181 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.670828 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.670833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.670969 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.671608 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:28.671627 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:28.672170 (+ 543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:28.673984 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:28.673989 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:28.673992 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:28.674220 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:28.674532 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:28.674535 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:28.674537 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:28.674625 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:28.675012 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:28.675015 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:28.675116 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 21:06:28.675224 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 21:06:28.675389 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":117,"threads_started":1}
Aug 06 21:06:28 dist-test-slave-69rg krb5kdc[30263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 21:06:28 dist-test-slave-69rg krb5kdc[30263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:28.704360 30360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.694095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42684 (local address 127.27.225.254:38963)
0806 21:06:28.694279 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:28.694285 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:28.694304 (+ 19us) server_negotiation.cc:407] Connection header received
0806 21:06:28.694364 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:28.694368 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:28.694432 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:28.694536 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:28.695380 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.696354 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.697193 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.697650 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.697774 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.700338 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:28.700343 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:28.700354 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:28.700402 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:28.702459 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:28.702880 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:28.702886 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:28.702891 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:28.702998 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:28.703267 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:28.703273 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:28.703277 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:28.703553 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:28.703719 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:28.703944 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 21:06:28.704097 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 21:06:28.708127 30311 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:06:28.713986 30311 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:06:28.733438 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.726148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:40909 (local address 127.0.0.1:48848)
0806 21:06:28.727372 (+ 1224us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:28.727388 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:28.727471 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:28.728095 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:28.728098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:28.728115 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:28.728212 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.728219 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.729566 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.729569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.731459 (+ 1890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.731471 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.732779 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.732783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.732921 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.732962 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 21:06:28.733076 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 21:06:28.733245 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1151,"mutex_wait_us":330,"thread_start_us":92,"threads_started":1}
I20260806 21:06:28.733497 30823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.724878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:42839 (local address 127.0.0.1:44740)
0806 21:06:28.726331 (+ 1453us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:28.726348 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:28.726467 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:28.727585 (+ 1118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:28.727590 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:28.727625 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:28.727773 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.727783 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.729585 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.729590 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.731491 (+ 1901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.731507 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.732869 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.732875 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.733018 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.733062 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 21:06:28.733115 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 21:06:28.733255 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":271,"spinlock_wait_cycles":8576,"thread_start_us":108,"threads_started":1}
I20260806 21:06:28.734112 30822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.725057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:39567 (local address 127.0.0.1:50090)
0806 21:06:28.726257 (+ 1200us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:28.726314 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:28.726467 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:28.727132 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:28.727137 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:28.727161 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:28.727313 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.727322 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.729545 (+ 2223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.729548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.732195 (+ 2647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.732205 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.733470 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.733475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.733748 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.733771 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 21:06:28.733904 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 21:06:28.733977 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1044,"mutex_wait_us":253,"spinlock_wait_cycles":4736,"thread_start_us":143,"threads_started":1}
I20260806 21:06:28.734452 30827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.726580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48848 (local address 127.27.225.193:40909)
0806 21:06:28.726892 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:28.726899 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:28.727564 (+ 665us) server_negotiation.cc:407] Connection header received
0806 21:06:28.727637 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:28.727642 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:28.727829 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:28.727959 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:28.728350 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.729451 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.731610 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.732467 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.732579 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.732713 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:28.734130 (+ 1417us) server_negotiation.cc:299] Negotiation successful
0806 21:06:28.734330 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":80,"threads_started":1}
I20260806 21:06:28.734459 30825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.725725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44740 (local address 127.27.225.195:42839)
0806 21:06:28.727095 (+ 1370us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:28.727108 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:28.727133 (+ 25us) server_negotiation.cc:407] Connection header received
0806 21:06:28.727218 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:28.727225 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:28.727414 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:28.727952 (+ 538us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:28.728120 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.729417 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.731688 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.732706 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.733924 (+ 1218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.734083 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:28.734141 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 21:06:28.734256 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1248,"thread_start_us":183,"threads_started":1}
I20260806 21:06:28.734622 30824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.725201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50090 (local address 127.27.225.194:39567)
0806 21:06:28.726234 (+ 1033us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:28.726245 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:28.726607 (+ 362us) server_negotiation.cc:407] Connection header received
0806 21:06:28.726689 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:28.726693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:28.726958 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:28.727124 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:28.727456 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.729404 (+ 1948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.732352 (+ 2948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.733139 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.733248 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.733369 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:28.734278 (+ 909us) server_negotiation.cc:299] Negotiation successful
0806 21:06:28.734376 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":932,"thread_start_us":133,"threads_started":1}
I20260806 21:06:28.737357 30451 tablet_service.cc:1467] Processing CreateTablet for tablet 2149b50ea62b4674a0324190098f3c07 (DEFAULT_TABLE table=test-table [id=db0f191ee82c4f0c931cdc649bff3efe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:28.737685 30732 tablet_service.cc:1467] Processing CreateTablet for tablet 2149b50ea62b4674a0324190098f3c07 (DEFAULT_TABLE table=test-table [id=db0f191ee82c4f0c931cdc649bff3efe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:28.738438 30451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2149b50ea62b4674a0324190098f3c07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:28.738858 30732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2149b50ea62b4674a0324190098f3c07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:28.744537 30593 tablet_service.cc:1467] Processing CreateTablet for tablet 2149b50ea62b4674a0324190098f3c07 (DEFAULT_TABLE table=test-table [id=db0f191ee82c4f0c931cdc649bff3efe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:28.745684 30593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2149b50ea62b4674a0324190098f3c07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:28.747649 30828 tablet_bootstrap.cc:492] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637: Bootstrap starting.
I20260806 21:06:28.749557 30829 tablet_bootstrap.cc:492] T 2149b50ea62b4674a0324190098f3c07 P a85bbe87cbd64819965452624929cdf5: Bootstrap starting.
I20260806 21:06:28.750174 30828 tablet_bootstrap.cc:654] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:28.751703 30828 log.cc:824] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:28.752100 30829 tablet_bootstrap.cc:654] T 2149b50ea62b4674a0324190098f3c07 P a85bbe87cbd64819965452624929cdf5: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:28.753146 30829 log.cc:824] T 2149b50ea62b4674a0324190098f3c07 P a85bbe87cbd64819965452624929cdf5: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:28.753961 30828 tablet_bootstrap.cc:492] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637: No bootstrap required, opened a new log
I20260806 21:06:28.754168 30828 ts_tablet_manager.cc:1397] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 21:06:28.755507 30829 tablet_bootstrap.cc:492] T 2149b50ea62b4674a0324190098f3c07 P a85bbe87cbd64819965452624929cdf5: No bootstrap required, opened a new log
I20260806 21:06:28.755752 30829 ts_tablet_manager.cc:1397] T 2149b50ea62b4674a0324190098f3c07 P a85bbe87cbd64819965452624929cdf5: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 21:06:28.758252 30832 tablet_bootstrap.cc:492] T 2149b50ea62b4674a0324190098f3c07 P 754e07fa5de64d52abb908bfa66dc766: Bootstrap starting.
I20260806 21:06:28.758657 30828 raft_consensus.cc:348] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } }
I20260806 21:06:28.758966 30828 raft_consensus.cc:374] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:28.759032 30828 raft_consensus.cc:729] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c660a9fb7174be199d18350f472d637, State: Initialized, Role: FOLLOWER
I20260806 21:06:28.759552 30828 consensus_queue.cc:260] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } }
I20260806 21:06:28.760543 30828 ts_tablet_manager.cc:1428] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 21:06:28.760924 30517 heartbeater.cc:503] Master 127.27.225.254:38963 was elected leader, sending a full tablet report...
I20260806 21:06:28.760926 30832 tablet_bootstrap.cc:654] T 2149b50ea62b4674a0324190098f3c07 P 754e07fa5de64d52abb908bfa66dc766: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:28.760951 30829 raft_consensus.cc:348] T 2149b50ea62b4674a0324190098f3c07 P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } }
I20260806 21:06:28.761258 30829 raft_consensus.cc:374] T 2149b50ea62b4674a0324190098f3c07 P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:28.761335 30829 raft_consensus.cc:729] T 2149b50ea62b4674a0324190098f3c07 P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a85bbe87cbd64819965452624929cdf5, State: Initialized, Role: FOLLOWER
I20260806 21:06:28.761890 30829 consensus_queue.cc:260] T 2149b50ea62b4674a0324190098f3c07 P a85bbe87cbd64819965452624929cdf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } }
I20260806 21:06:28.762173 30832 log.cc:824] T 2149b50ea62b4674a0324190098f3c07 P 754e07fa5de64d52abb908bfa66dc766: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:28.763094 30829 ts_tablet_manager.cc:1428] T 2149b50ea62b4674a0324190098f3c07 P a85bbe87cbd64819965452624929cdf5: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 21:06:28.763492 30804 heartbeater.cc:503] Master 127.27.225.254:38963 was elected leader, sending a full tablet report...
I20260806 21:06:28.765950 30832 tablet_bootstrap.cc:492] T 2149b50ea62b4674a0324190098f3c07 P 754e07fa5de64d52abb908bfa66dc766: No bootstrap required, opened a new log
I20260806 21:06:28.766165 30832 ts_tablet_manager.cc:1397] T 2149b50ea62b4674a0324190098f3c07 P 754e07fa5de64d52abb908bfa66dc766: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 21:06:28.769335 30832 raft_consensus.cc:348] T 2149b50ea62b4674a0324190098f3c07 P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } }
I20260806 21:06:28.769565 30832 raft_consensus.cc:374] T 2149b50ea62b4674a0324190098f3c07 P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:28.769596 30832 raft_consensus.cc:729] T 2149b50ea62b4674a0324190098f3c07 P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 754e07fa5de64d52abb908bfa66dc766, State: Initialized, Role: FOLLOWER
I20260806 21:06:28.770064 30832 consensus_queue.cc:260] T 2149b50ea62b4674a0324190098f3c07 P 754e07fa5de64d52abb908bfa66dc766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } }
I20260806 21:06:28.770813 30661 heartbeater.cc:503] Master 127.27.225.254:38963 was elected leader, sending a full tablet report...
I20260806 21:06:28.771188 30832 ts_tablet_manager.cc:1428] T 2149b50ea62b4674a0324190098f3c07 P 754e07fa5de64d52abb908bfa66dc766: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 21:06:28.781780 30833 raft_consensus.cc:482] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:06:28.782075 30833 raft_consensus.cc:504] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } }
I20260806 21:06:28.783900 30833 leader_election.cc:288] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a85bbe87cbd64819965452624929cdf5 (127.27.225.195:42839), 754e07fa5de64d52abb908bfa66dc766 (127.27.225.194:39567)
I20260806 21:06:28.789165 30824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.784323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:52305 (local address 127.27.225.194:39567)
0806 21:06:28.784487 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:28.784495 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:28.784801 (+ 306us) server_negotiation.cc:407] Connection header received
0806 21:06:28.784902 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:28.784908 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:28.784977 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:28.785087 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:28.785475 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.786054 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.788197 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.788598 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.788731 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.788784 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:28.788968 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 21:06:28.789038 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 21:06:28.789187 30838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.784110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:39567 (local address 127.27.225.193:52305)
0806 21:06:28.784704 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:28.784716 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:28.784801 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:28.785193 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:28.785198 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:28.785211 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:28.785311 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.785318 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.786168 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.786171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.788086 (+ 1915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.788093 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.788724 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.788727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.788832 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.788854 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 21:06:28.788953 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 21:06:28.789029 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"mutex_wait_us":353,"thread_start_us":90,"threads_started":1}
I20260806 21:06:28.789947 30825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.784381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:32973 (local address 127.27.225.195:42839)
0806 21:06:28.784648 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:28.784655 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:28.784675 (+ 20us) server_negotiation.cc:407] Connection header received
0806 21:06:28.784929 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:28.784931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:28.784971 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:28.785065 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:28.785401 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.786527 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.788039 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.788969 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.789158 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.789237 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:28.789721 (+ 484us) server_negotiation.cc:299] Negotiation successful
0806 21:06:28.789810 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260806 21:06:28.790045 30837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.784110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:42839 (local address 127.27.225.193:32973)
0806 21:06:28.784587 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:28.784603 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:28.784720 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:28.785076 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:28.785079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:28.785096 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:28.785234 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.785241 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.786698 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.786701 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.787883 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.787895 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.789345 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.789351 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.789506 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.789545 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 21:06:28.789694 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 21:06:28.789799 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":6,"thread_start_us":153,"threads_started":1}
I20260806 21:06:28.790326 30613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2149b50ea62b4674a0324190098f3c07" candidate_uuid: "4c660a9fb7174be199d18350f472d637" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "754e07fa5de64d52abb908bfa66dc766" is_pre_election: true
I20260806 21:06:28.790678 30613 raft_consensus.cc:2454] T 2149b50ea62b4674a0324190098f3c07 P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c660a9fb7174be199d18350f472d637 in term 0.
I20260806 21:06:28.790680 30756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2149b50ea62b4674a0324190098f3c07" candidate_uuid: "4c660a9fb7174be199d18350f472d637" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a85bbe87cbd64819965452624929cdf5" is_pre_election: true
I20260806 21:06:28.790956 30756 raft_consensus.cc:2454] T 2149b50ea62b4674a0324190098f3c07 P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c660a9fb7174be199d18350f472d637 in term 0.
I20260806 21:06:28.791217 30399 leader_election.cc:302] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c660a9fb7174be199d18350f472d637, 754e07fa5de64d52abb908bfa66dc766; no voters:
I20260806 21:06:28.791499 30833 raft_consensus.cc:2781] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:06:28.791621 30833 raft_consensus.cc:482] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:06:28.791669 30833 raft_consensus.cc:3037] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:28.792886 30833 raft_consensus.cc:504] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } }
I20260806 21:06:28.793267 30833 leader_election.cc:288] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637 [CANDIDATE]: Term 1 election: Requested vote from peers a85bbe87cbd64819965452624929cdf5 (127.27.225.195:42839), 754e07fa5de64d52abb908bfa66dc766 (127.27.225.194:39567)
I20260806 21:06:28.793639 30756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2149b50ea62b4674a0324190098f3c07" candidate_uuid: "4c660a9fb7174be199d18350f472d637" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a85bbe87cbd64819965452624929cdf5"
I20260806 21:06:28.793639 30613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2149b50ea62b4674a0324190098f3c07" candidate_uuid: "4c660a9fb7174be199d18350f472d637" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "754e07fa5de64d52abb908bfa66dc766"
I20260806 21:06:28.793769 30613 raft_consensus.cc:3037] T 2149b50ea62b4674a0324190098f3c07 P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:28.793769 30756 raft_consensus.cc:3037] T 2149b50ea62b4674a0324190098f3c07 P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:28.795143 30613 raft_consensus.cc:2454] T 2149b50ea62b4674a0324190098f3c07 P 754e07fa5de64d52abb908bfa66dc766 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c660a9fb7174be199d18350f472d637 in term 1.
I20260806 21:06:28.795164 30756 raft_consensus.cc:2454] T 2149b50ea62b4674a0324190098f3c07 P a85bbe87cbd64819965452624929cdf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c660a9fb7174be199d18350f472d637 in term 1.
I20260806 21:06:28.795495 30401 leader_election.cc:302] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c660a9fb7174be199d18350f472d637, a85bbe87cbd64819965452624929cdf5; no voters:
I20260806 21:06:28.795734 30833 raft_consensus.cc:2781] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:28.796017 30833 raft_consensus.cc:686] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637 [term 1 LEADER]: Becoming Leader. State: Replica: 4c660a9fb7174be199d18350f472d637, State: Running, Role: LEADER
I20260806 21:06:28.796595 30833 consensus_queue.cc:237] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } }
I20260806 21:06:28.801025 30311 catalog_manager.cc:5495] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c660a9fb7174be199d18350f472d637 (127.27.225.193). New cstate: current_term: 1 leader_uuid: "4c660a9fb7174be199d18350f472d637" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } health_report { overall_health: HEALTHY } } }
Aug 06 21:06:28 dist-test-slave-69rg krb5kdc[30263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050387, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:28.852226 30360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.843723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42698 (local address 127.27.225.254:38963)
0806 21:06:28.843873 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:28.843877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:28.844067 (+ 190us) server_negotiation.cc:407] Connection header received
0806 21:06:28.844193 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:28.844195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:28.844236 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:28.844317 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:28.844972 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.845490 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.846397 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.846661 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.846740 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.849131 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:28.849134 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:28.849136 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:28.849157 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:28.850670 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:28.851102 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:28.851105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:28.851106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:28.851176 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:28.851417 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:28.851419 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:28.851420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:28.851584 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:28.851657 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:28.851940 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 21:06:28.852037 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 21:06:28.852229 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.843639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:38963 (local address 127.0.0.1:42698)
0806 21:06:28.843964 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:28.843982 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:28.844102 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:28.844355 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:28.844357 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:28.844765 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:28.844862 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.844868 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.845618 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.845620 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.846279 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.846289 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.846760 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.846763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.846823 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.847409 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:28.847421 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:28.848998 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:28.850771 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:28.850780 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:28.850785 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:28.851018 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:28.851260 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:28.851262 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:28.851264 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:28.851341 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:28.851678 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:28.851684 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:28.851783 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 21:06:28.851873 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 21:06:28.851986 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":142,"threads_started":1}
I20260806 21:06:28.856665 30311 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:42698:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 21:06:28.857496 30311 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:06:28.862550 30593 tablet_service.cc:1467] Processing CreateTablet for tablet a00355c2af9145ce96f8c599067ee75f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5892c5747d0e4f699241b468f24f98cc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 21:06:28.862511 30451 tablet_service.cc:1467] Processing CreateTablet for tablet a00355c2af9145ce96f8c599067ee75f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5892c5747d0e4f699241b468f24f98cc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 21:06:28.862823 30732 tablet_service.cc:1467] Processing CreateTablet for tablet a00355c2af9145ce96f8c599067ee75f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5892c5747d0e4f699241b468f24f98cc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 21:06:28.863355 30593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a00355c2af9145ce96f8c599067ee75f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:28.863401 30732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a00355c2af9145ce96f8c599067ee75f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:28.863401 30451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a00355c2af9145ce96f8c599067ee75f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:28.866381 30832 tablet_bootstrap.cc:492] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766: Bootstrap starting.
I20260806 21:06:28.866407 30829 tablet_bootstrap.cc:492] T a00355c2af9145ce96f8c599067ee75f P a85bbe87cbd64819965452624929cdf5: Bootstrap starting.
I20260806 21:06:28.867004 30828 tablet_bootstrap.cc:492] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637: Bootstrap starting.
I20260806 21:06:28.867408 30832 tablet_bootstrap.cc:654] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:28.867462 30829 tablet_bootstrap.cc:654] T a00355c2af9145ce96f8c599067ee75f P a85bbe87cbd64819965452624929cdf5: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:28.868474 30828 tablet_bootstrap.cc:654] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:28.868691 30832 tablet_bootstrap.cc:492] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766: No bootstrap required, opened a new log
I20260806 21:06:28.868779 30832 ts_tablet_manager.cc:1397] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 21:06:28.868798 30829 tablet_bootstrap.cc:492] T a00355c2af9145ce96f8c599067ee75f P a85bbe87cbd64819965452624929cdf5: No bootstrap required, opened a new log
I20260806 21:06:28.868885 30829 ts_tablet_manager.cc:1397] T a00355c2af9145ce96f8c599067ee75f P a85bbe87cbd64819965452624929cdf5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:06:28.869294 30832 raft_consensus.cc:348] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:28.869345 30829 raft_consensus.cc:348] T a00355c2af9145ce96f8c599067ee75f P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:28.869423 30832 raft_consensus.cc:374] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:28.869454 30829 raft_consensus.cc:374] T a00355c2af9145ce96f8c599067ee75f P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:28.869467 30832 raft_consensus.cc:729] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 754e07fa5de64d52abb908bfa66dc766, State: Initialized, Role: FOLLOWER
I20260806 21:06:28.869501 30829 raft_consensus.cc:729] T a00355c2af9145ce96f8c599067ee75f P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a85bbe87cbd64819965452624929cdf5, State: Initialized, Role: FOLLOWER
I20260806 21:06:28.869611 30832 consensus_queue.cc:260] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:28.869653 30829 consensus_queue.cc:260] T a00355c2af9145ce96f8c599067ee75f P a85bbe87cbd64819965452624929cdf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:28.869928 30832 ts_tablet_manager.cc:1428] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 21:06:28.869963 30829 ts_tablet_manager.cc:1428] T a00355c2af9145ce96f8c599067ee75f P a85bbe87cbd64819965452624929cdf5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:06:28.870139 30828 tablet_bootstrap.cc:492] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637: No bootstrap required, opened a new log
I20260806 21:06:28.870239 30828 ts_tablet_manager.cc:1397] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:06:28.870213 30836 tablet_replica.cc:439] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } } }
I20260806 21:06:28.870375 30836 tablet_replica.cc:442] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:06:28.870574 30834 tablet_replica.cc:439] T a00355c2af9145ce96f8c599067ee75f P a85bbe87cbd64819965452624929cdf5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } } }
I20260806 21:06:28.870715 30834 tablet_replica.cc:442] T a00355c2af9145ce96f8c599067ee75f P a85bbe87cbd64819965452624929cdf5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:06:28.870711 30828 raft_consensus.cc:348] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:28.870824 30828 raft_consensus.cc:374] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:28.870867 30828 raft_consensus.cc:729] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c660a9fb7174be199d18350f472d637, State: Initialized, Role: FOLLOWER
I20260806 21:06:28.870988 30828 consensus_queue.cc:260] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:28.871244 30833 tablet_replica.cc:439] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } } }
I20260806 21:06:28.871382 30833 tablet_replica.cc:442] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:06:28.871253 30828 ts_tablet_manager.cc:1428] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260806 21:06:28.891532 30518 tablet.cc:2369] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 21:06:28.902983 30662 tablet.cc:2369] T 2149b50ea62b4674a0324190098f3c07 P 754e07fa5de64d52abb908bfa66dc766: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 21:06:28.909515 30805 tablet.cc:2369] T 2149b50ea62b4674a0324190098f3c07 P a85bbe87cbd64819965452624929cdf5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:06:28.969877 30836 raft_consensus.cc:482] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:06:28.970034 30836 raft_consensus.cc:504] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:28.971226 30836 leader_election.cc:288] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c660a9fb7174be199d18350f472d637 (127.27.225.193:40909), a85bbe87cbd64819965452624929cdf5 (127.27.225.195:42839)
I20260806 21:06:28.973500 30833 raft_consensus.cc:482] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:06:28.973634 30833 raft_consensus.cc:504] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:28.974047 30833 leader_election.cc:288] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 754e07fa5de64d52abb908bfa66dc766 (127.27.225.194:39567), a85bbe87cbd64819965452624929cdf5 (127.27.225.195:42839)
I20260806 21:06:28.974416 30613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a00355c2af9145ce96f8c599067ee75f" candidate_uuid: "4c660a9fb7174be199d18350f472d637" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "754e07fa5de64d52abb908bfa66dc766" is_pre_election: true
I20260806 21:06:28.974491 30756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a00355c2af9145ce96f8c599067ee75f" candidate_uuid: "4c660a9fb7174be199d18350f472d637" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a85bbe87cbd64819965452624929cdf5" is_pre_election: true
I20260806 21:06:28.974565 30613 raft_consensus.cc:2454] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c660a9fb7174be199d18350f472d637 in term 0.
I20260806 21:06:28.974643 30756 raft_consensus.cc:2454] T a00355c2af9145ce96f8c599067ee75f P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c660a9fb7174be199d18350f472d637 in term 0.
I20260806 21:06:28.974939 30401 leader_election.cc:302] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c660a9fb7174be199d18350f472d637, a85bbe87cbd64819965452624929cdf5; no voters:
I20260806 21:06:28.975183 30833 raft_consensus.cc:2781] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:06:28.975247 30833 raft_consensus.cc:482] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:06:28.975286 30833 raft_consensus.cc:3037] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:28.975939 30827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.971661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:55271 (local address 127.27.225.193:40909)
0806 21:06:28.971889 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:28.971893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:28.971912 (+ 19us) server_negotiation.cc:407] Connection header received
0806 21:06:28.972087 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:28.972090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:28.972132 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:28.972219 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:28.972546 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.973069 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.974539 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.975123 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.975223 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.975283 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:28.975754 (+ 471us) server_negotiation.cc:299] Negotiation successful
0806 21:06:28.975827 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260806 21:06:28.975936 30825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.971609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:58367 (local address 127.27.225.195:42839)
0806 21:06:28.971755 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:28.971762 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:28.972017 (+ 255us) server_negotiation.cc:407] Connection header received
0806 21:06:28.972153 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:28.972156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:28.972205 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:28.972311 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:28.972640 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.973133 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.974543 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.975110 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.975202 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.975294 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:28.975744 (+ 450us) server_negotiation.cc:299] Negotiation successful
0806 21:06:28.975826 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 21:06:28.975948 30851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.971514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:42839 (local address 127.27.225.194:58367)
0806 21:06:28.971907 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:28.971922 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:28.972038 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:28.972361 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:28.972364 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:28.972381 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:28.972515 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.972521 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.973244 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.973250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.974395 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.974404 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.975395 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.975398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.975617 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.975636 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 21:06:28.975724 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 21:06:28.975802 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":87,"thread_start_us":82,"threads_started":1}
I20260806 21:06:28.976330 30850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.971514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:40909 (local address 127.27.225.194:55271)
0806 21:06:28.971846 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:28.971859 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:28.971950 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:28.972232 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:28.972235 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:28.972250 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:28.972397 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.972406 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.973214 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.973217 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.974394 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.974412 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.975386 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.975390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.975597 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.975631 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 21:06:28.976112 (+ 481us) client_negotiation.cc:240] Negotiation successful
0806 21:06:28.976188 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":53,"thread_start_us":90,"threads_started":1}
I20260806 21:06:28.976559 30756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a00355c2af9145ce96f8c599067ee75f" candidate_uuid: "754e07fa5de64d52abb908bfa66dc766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a85bbe87cbd64819965452624929cdf5" is_pre_election: true
I20260806 21:06:28.976698 30756 raft_consensus.cc:2454] T a00355c2af9145ce96f8c599067ee75f P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 754e07fa5de64d52abb908bfa66dc766 in term 0.
I20260806 21:06:28.976656 30833 raft_consensus.cc:504] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:28.977039 30833 leader_election.cc:288] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637 [CANDIDATE]: Term 1 election: Requested vote from peers 754e07fa5de64d52abb908bfa66dc766 (127.27.225.194:39567), a85bbe87cbd64819965452624929cdf5 (127.27.225.195:42839)
I20260806 21:06:28.977042 30546 leader_election.cc:302] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 754e07fa5de64d52abb908bfa66dc766, a85bbe87cbd64819965452624929cdf5; no voters:
I20260806 21:06:28.977383 30836 raft_consensus.cc:2781] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:06:28.977447 30836 raft_consensus.cc:482] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:06:28.977445 30756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a00355c2af9145ce96f8c599067ee75f" candidate_uuid: "4c660a9fb7174be199d18350f472d637" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a85bbe87cbd64819965452624929cdf5"
I20260806 21:06:28.977489 30836 raft_consensus.cc:3037] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:28.977504 30471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a00355c2af9145ce96f8c599067ee75f" candidate_uuid: "754e07fa5de64d52abb908bfa66dc766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c660a9fb7174be199d18350f472d637" is_pre_election: true
I20260806 21:06:28.977550 30756 raft_consensus.cc:3037] T a00355c2af9145ce96f8c599067ee75f P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:28.977695 30471 raft_consensus.cc:2379] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 754e07fa5de64d52abb908bfa66dc766 in current term 1: Already voted for candidate 4c660a9fb7174be199d18350f472d637 in this term.
I20260806 21:06:28.978195 30613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a00355c2af9145ce96f8c599067ee75f" candidate_uuid: "4c660a9fb7174be199d18350f472d637" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "754e07fa5de64d52abb908bfa66dc766"
I20260806 21:06:28.978698 30756 raft_consensus.cc:2454] T a00355c2af9145ce96f8c599067ee75f P a85bbe87cbd64819965452624929cdf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c660a9fb7174be199d18350f472d637 in term 1.
I20260806 21:06:28.978703 30836 raft_consensus.cc:504] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:28.978978 30613 raft_consensus.cc:2379] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4c660a9fb7174be199d18350f472d637 in current term 1: Already voted for candidate 754e07fa5de64d52abb908bfa66dc766 in this term.
I20260806 21:06:28.979058 30401 leader_election.cc:302] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c660a9fb7174be199d18350f472d637, a85bbe87cbd64819965452624929cdf5; no voters:
I20260806 21:06:28.979396 30471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a00355c2af9145ce96f8c599067ee75f" candidate_uuid: "754e07fa5de64d52abb908bfa66dc766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c660a9fb7174be199d18350f472d637"
I20260806 21:06:28.979452 30833 raft_consensus.cc:2781] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:28.979439 30756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a00355c2af9145ce96f8c599067ee75f" candidate_uuid: "754e07fa5de64d52abb908bfa66dc766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a85bbe87cbd64819965452624929cdf5"
I20260806 21:06:28.979534 30833 raft_consensus.cc:686] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637 [term 1 LEADER]: Becoming Leader. State: Replica: 4c660a9fb7174be199d18350f472d637, State: Running, Role: LEADER
I20260806 21:06:28.979626 30756 raft_consensus.cc:2379] T a00355c2af9145ce96f8c599067ee75f P a85bbe87cbd64819965452624929cdf5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 754e07fa5de64d52abb908bfa66dc766 in current term 1: Already voted for candidate 4c660a9fb7174be199d18350f472d637 in this term.
I20260806 21:06:28.979674 30833 consensus_queue.cc:237] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:28.979889 30836 leader_election.cc:288] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766 [CANDIDATE]: Term 1 election: Requested vote from peers 4c660a9fb7174be199d18350f472d637 (127.27.225.193:40909), a85bbe87cbd64819965452624929cdf5 (127.27.225.195:42839)
I20260806 21:06:28.980454 30839 tablet_replica.cc:439] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637: TxnStatusTablet state changed. Reason: New leader 4c660a9fb7174be199d18350f472d637. Latest consensus state: current_term: 1 leader_uuid: "4c660a9fb7174be199d18350f472d637" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } } }
I20260806 21:06:28.980576 30839 tablet_replica.cc:442] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637: This TxnStatusTablet replica's current role is: LEADER
I20260806 21:06:28.980671 30544 leader_election.cc:302] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 754e07fa5de64d52abb908bfa66dc766; no voters: 4c660a9fb7174be199d18350f472d637, a85bbe87cbd64819965452624929cdf5
I20260806 21:06:28.980888 30836 raft_consensus.cc:2726] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260806 21:06:28.981052 30853 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 21:06:28.981274 30853 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 21:06:28.981612 30309 catalog_manager.cc:5495] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c660a9fb7174be199d18350f472d637 (127.27.225.193). New cstate: current_term: 1 leader_uuid: "4c660a9fb7174be199d18350f472d637" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } health_report { overall_health: UNKNOWN } } }
I20260806 21:06:29.001230 30827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.996688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48858 (local address 127.27.225.193:40909)
0806 21:06:28.996834 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:28.996840 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:28.996857 (+ 17us) server_negotiation.cc:407] Connection header received
0806 21:06:28.997047 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:28.997050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:28.997122 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:28.997227 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:06:28.997520 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.998353 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.999353 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.999716 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.999825 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:29.000144 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:06:29.000745 (+ 601us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:06:29.000860 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:29.001010 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 21:06:29.001073 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 21:06:29.001309 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:28.996570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:40909 (local address 127.0.0.1:48858)
0806 21:06:28.996784 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:28.996799 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:28.996898 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:28.997270 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:28.997273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:28.997281 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 21:06:28.997396 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.997401 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.998488 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.998491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.999184 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:28.999198 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:28.999853 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:28.999856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:28.999937 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:28.999989 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 21:06:29.000872 (+ 883us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 21:06:29.000903 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 21:06:29.000971 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 21:06:29.001104 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 21:06:29.006855 30756 raft_consensus.cc:1261] T a00355c2af9145ce96f8c599067ee75f P a85bbe87cbd64819965452624929cdf5 [term 1 FOLLOWER]: Refusing update from remote peer 4c660a9fb7174be199d18350f472d637: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:06:29.006865 30613 raft_consensus.cc:1261] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766 [term 1 FOLLOWER]: Refusing update from remote peer 4c660a9fb7174be199d18350f472d637: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:06:29.007508 30833 consensus_queue.cc:1035] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:06:29.007726 30839 consensus_queue.cc:1035] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637 [LEADER]: Connected to new peer: Peer: permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:06:29.011633 30834 tablet_replica.cc:439] T a00355c2af9145ce96f8c599067ee75f P a85bbe87cbd64819965452624929cdf5: TxnStatusTablet state changed. Reason: New leader 4c660a9fb7174be199d18350f472d637. Latest consensus state: current_term: 1 leader_uuid: "4c660a9fb7174be199d18350f472d637" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } } }
I20260806 21:06:29.011794 30834 tablet_replica.cc:442] T a00355c2af9145ce96f8c599067ee75f P a85bbe87cbd64819965452624929cdf5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:06:29.012063 30836 tablet_replica.cc:439] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766: TxnStatusTablet state changed. Reason: New leader 4c660a9fb7174be199d18350f472d637. Latest consensus state: current_term: 1 leader_uuid: "4c660a9fb7174be199d18350f472d637" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } } }
I20260806 21:06:29.012225 30836 tablet_replica.cc:442] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:06:29.015897 30833 tablet_replica.cc:439] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4c660a9fb7174be199d18350f472d637" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } } }
I20260806 21:06:29.016075 30833 tablet_replica.cc:442] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637: This TxnStatusTablet replica's current role is: LEADER
I20260806 21:06:29.016453 30834 tablet_replica.cc:439] T a00355c2af9145ce96f8c599067ee75f P a85bbe87cbd64819965452624929cdf5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4c660a9fb7174be199d18350f472d637" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } } }
I20260806 21:06:29.016579 30834 tablet_replica.cc:442] T a00355c2af9145ce96f8c599067ee75f P a85bbe87cbd64819965452624929cdf5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:06:29.016968 30856 mvcc.cc:205] Tried to move back new op lower bound from 7315662393364463616 to 7315662393261842432. Current Snapshot: MvccSnapshot[applied={T|T < 7315662393364463616}]
I20260806 21:06:29.023991 30839 tablet_replica.cc:439] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4c660a9fb7174be199d18350f472d637" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } } }
I20260806 21:06:29.024159 30839 tablet_replica.cc:442] T a00355c2af9145ce96f8c599067ee75f P 4c660a9fb7174be199d18350f472d637: This TxnStatusTablet replica's current role is: LEADER
I20260806 21:06:29.025928 30836 tablet_replica.cc:439] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4c660a9fb7174be199d18350f472d637" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } } }
I20260806 21:06:29.026072 30836 tablet_replica.cc:442] T a00355c2af9145ce96f8c599067ee75f P 754e07fa5de64d52abb908bfa66dc766: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:06:29.026837 30827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:29.022618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48860 (local address 127.27.225.193:40909)
0806 21:06:29.022750 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:29.022755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:29.022769 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:06:29.022896 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:29.022900 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:29.022953 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:29.023033 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:06:29.023303 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:29.024062 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:29.025225 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:29.025512 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:29.025637 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:29.026068 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:06:29.026159 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:06:29.026224 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:29.026588 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 21:06:29.026683 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 21:06:29.026808 30858 mvcc.cc:205] Tried to move back new op lower bound from 7315662393364463616 to 7315662393261842432. Current Snapshot: MvccSnapshot[applied={T|T < 7315662393364463616}]
I20260806 21:06:29.039022 30827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:29.035622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:40699 (local address 127.27.225.193:40909)
0806 21:06:29.035767 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:29.035772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:29.035889 (+ 117us) server_negotiation.cc:407] Connection header received
0806 21:06:29.036048 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:29.036052 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:29.036111 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:29.036194 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:06:29.036468 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:29.036968 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:29.037869 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:29.038090 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:29.038211 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:29.038551 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:06:29.038620 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:06:29.038705 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:29.038790 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 21:06:29.038839 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":79}
I20260806 21:06:29.039032 30869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:29.035475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:40909 (local address 127.27.225.193:40699)
0806 21:06:29.035793 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:29.035815 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:29.035951 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:29.036234 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:29.036237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:29.036244 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 21:06:29.036366 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:29.036371 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:29.037090 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:29.037094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:29.037696 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:29.037704 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:29.038221 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:29.038224 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:29.038344 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:29.038386 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 21:06:29.038706 (+ 320us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 21:06:29.038712 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 21:06:29.038775 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 21:06:29.038857 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":4,"thread_start_us":96,"threads_started":1}
I20260806 21:06:29.046228 30613 raft_consensus.cc:1261] T 2149b50ea62b4674a0324190098f3c07 P 754e07fa5de64d52abb908bfa66dc766 [term 1 FOLLOWER]: Refusing update from remote peer 4c660a9fb7174be199d18350f472d637: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:06:29.046326 30756 raft_consensus.cc:1261] T 2149b50ea62b4674a0324190098f3c07 P a85bbe87cbd64819965452624929cdf5 [term 1 FOLLOWER]: Refusing update from remote peer 4c660a9fb7174be199d18350f472d637: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:06:29.046722 30833 consensus_queue.cc:1035] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637 [LEADER]: Connected to new peer: Peer: permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:06:29.046977 30865 consensus_queue.cc:1035] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:06:29.053258 30859 mvcc.cc:205] Tried to move back new op lower bound from 7315662393529053184 to 7315662392513863680. Current Snapshot: MvccSnapshot[applied={T|T < 7315662393529053184}]
I20260806 21:06:29.098183 30310 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:06:29.098376 30310 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:06:29.101441 30310 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de: Sending DeleteTablet for 3 replicas of tablet 2149b50ea62b4674a0324190098f3c07
I20260806 21:06:29.102486 30732 tablet_service.cc:1514] Processing DeleteTablet for tablet 2149b50ea62b4674a0324190098f3c07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:29 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:44740
I20260806 21:06:29.102535 30593 tablet_service.cc:1514] Processing DeleteTablet for tablet 2149b50ea62b4674a0324190098f3c07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:29 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:50090
I20260806 21:06:29.103221 30874 tablet_replica.cc:330] T 2149b50ea62b4674a0324190098f3c07 P 754e07fa5de64d52abb908bfa66dc766: stopping tablet replica
I20260806 21:06:29.103148 30310 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 21:06:29.103305 30873 tablet_replica.cc:330] T 2149b50ea62b4674a0324190098f3c07 P a85bbe87cbd64819965452624929cdf5: stopping tablet replica
I20260806 21:06:29.103569 30873 raft_consensus.cc:2229] T 2149b50ea62b4674a0324190098f3c07 P a85bbe87cbd64819965452624929cdf5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:06:29.103569 30874 raft_consensus.cc:2229] T 2149b50ea62b4674a0324190098f3c07 P 754e07fa5de64d52abb908bfa66dc766 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 21:06:29.103595 30310 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:06:29.103789 30874 raft_consensus.cc:2258] T 2149b50ea62b4674a0324190098f3c07 P 754e07fa5de64d52abb908bfa66dc766 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:29.103789 30873 raft_consensus.cc:2258] T 2149b50ea62b4674a0324190098f3c07 P a85bbe87cbd64819965452624929cdf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:29.105298 30874 ts_tablet_manager.cc:1905] T 2149b50ea62b4674a0324190098f3c07 P 754e07fa5de64d52abb908bfa66dc766: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:29.105298 30873 ts_tablet_manager.cc:1905] T 2149b50ea62b4674a0324190098f3c07 P a85bbe87cbd64819965452624929cdf5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:29.106248 30450 tablet_service.cc:1514] Processing DeleteTablet for tablet 2149b50ea62b4674a0324190098f3c07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:29 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:48848
I20260806 21:06:29.108544 30874 ts_tablet_manager.cc:1918] T 2149b50ea62b4674a0324190098f3c07 P 754e07fa5de64d52abb908bfa66dc766: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:06:29.108544 30873 ts_tablet_manager.cc:1918] T 2149b50ea62b4674a0324190098f3c07 P a85bbe87cbd64819965452624929cdf5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:06:29.108649 30873 log.cc:1196] T 2149b50ea62b4674a0324190098f3c07 P a85bbe87cbd64819965452624929cdf5: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/2149b50ea62b4674a0324190098f3c07
I20260806 21:06:29.108649 30874 log.cc:1196] T 2149b50ea62b4674a0324190098f3c07 P 754e07fa5de64d52abb908bfa66dc766: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-1/wal/wals/2149b50ea62b4674a0324190098f3c07
I20260806 21:06:29.108987 30874 ts_tablet_manager.cc:1939] T 2149b50ea62b4674a0324190098f3c07 P 754e07fa5de64d52abb908bfa66dc766: Deleting consensus metadata
I20260806 21:06:29.109005 30873 ts_tablet_manager.cc:1939] T 2149b50ea62b4674a0324190098f3c07 P a85bbe87cbd64819965452624929cdf5: Deleting consensus metadata
I20260806 21:06:29.110404 30875 tablet_replica.cc:330] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637: stopping tablet replica
I20260806 21:06:29.110592 30295 catalog_manager.cc:4841] TS 754e07fa5de64d52abb908bfa66dc766 (127.27.225.194:39567): tablet 2149b50ea62b4674a0324190098f3c07 (table test-table [id=db0f191ee82c4f0c931cdc649bff3efe]) successfully deleted
I20260806 21:06:29.110739 30593 tablet_service.cc:1467] Processing CreateTablet for tablet 1425e2f04b36418aab14b521e6547f70 (DEFAULT_TABLE table=test-table [id=292af2dd8fd5456983a70bb05e64b5cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:29.110797 30875 raft_consensus.cc:2229] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:06:29.111001 30593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1425e2f04b36418aab14b521e6547f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:29.111037 30450 tablet_service.cc:1467] Processing CreateTablet for tablet 1425e2f04b36418aab14b521e6547f70 (DEFAULT_TABLE table=test-table [id=292af2dd8fd5456983a70bb05e64b5cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:29.111155 30875 raft_consensus.cc:2258] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:29.111301 30450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1425e2f04b36418aab14b521e6547f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:29.112656 30875 ts_tablet_manager.cc:1905] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:29.113490 30832 tablet_bootstrap.cc:492] T 1425e2f04b36418aab14b521e6547f70 P 754e07fa5de64d52abb908bfa66dc766: Bootstrap starting.
I20260806 21:06:29.113961 30828 tablet_bootstrap.cc:492] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637: Bootstrap starting.
I20260806 21:06:29.114472 30832 tablet_bootstrap.cc:654] T 1425e2f04b36418aab14b521e6547f70 P 754e07fa5de64d52abb908bfa66dc766: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:29.114614 30297 catalog_manager.cc:4841] TS a85bbe87cbd64819965452624929cdf5 (127.27.225.195:42839): tablet 2149b50ea62b4674a0324190098f3c07 (table test-table [id=db0f191ee82c4f0c931cdc649bff3efe]) successfully deleted
I20260806 21:06:29.114928 30828 tablet_bootstrap.cc:654] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:29.115211 30732 tablet_service.cc:1467] Processing CreateTablet for tablet 1425e2f04b36418aab14b521e6547f70 (DEFAULT_TABLE table=test-table [id=292af2dd8fd5456983a70bb05e64b5cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:29.115507 30732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1425e2f04b36418aab14b521e6547f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:29.115847 30875 ts_tablet_manager.cc:1918] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:06:29.115971 30875 log.cc:1196] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-0/wal/wals/2149b50ea62b4674a0324190098f3c07
I20260806 21:06:29.116370 30875 ts_tablet_manager.cc:1939] T 2149b50ea62b4674a0324190098f3c07 P 4c660a9fb7174be199d18350f472d637: Deleting consensus metadata
I20260806 21:06:29.116700 30832 tablet_bootstrap.cc:492] T 1425e2f04b36418aab14b521e6547f70 P 754e07fa5de64d52abb908bfa66dc766: No bootstrap required, opened a new log
I20260806 21:06:29.116791 30832 ts_tablet_manager.cc:1397] T 1425e2f04b36418aab14b521e6547f70 P 754e07fa5de64d52abb908bfa66dc766: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:06:29.117233 30832 raft_consensus.cc:348] T 1425e2f04b36418aab14b521e6547f70 P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:29.117352 30832 raft_consensus.cc:374] T 1425e2f04b36418aab14b521e6547f70 P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:29.117384 30832 raft_consensus.cc:729] T 1425e2f04b36418aab14b521e6547f70 P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 754e07fa5de64d52abb908bfa66dc766, State: Initialized, Role: FOLLOWER
I20260806 21:06:29.117509 30832 consensus_queue.cc:260] T 1425e2f04b36418aab14b521e6547f70 P 754e07fa5de64d52abb908bfa66dc766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:29.117799 30832 ts_tablet_manager.cc:1428] T 1425e2f04b36418aab14b521e6547f70 P 754e07fa5de64d52abb908bfa66dc766: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:06:29.118737 30828 tablet_bootstrap.cc:492] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637: No bootstrap required, opened a new log
I20260806 21:06:29.118749 30829 tablet_bootstrap.cc:492] T 1425e2f04b36418aab14b521e6547f70 P a85bbe87cbd64819965452624929cdf5: Bootstrap starting.
I20260806 21:06:29.118825 30828 ts_tablet_manager.cc:1397] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260806 21:06:29.119210 30295 catalog_manager.cc:4841] TS 4c660a9fb7174be199d18350f472d637 (127.27.225.193:40909): tablet 2149b50ea62b4674a0324190098f3c07 (table test-table [id=db0f191ee82c4f0c931cdc649bff3efe]) successfully deleted
I20260806 21:06:29.119308 30828 raft_consensus.cc:348] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:29.119506 30828 raft_consensus.cc:374] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:29.119549 30828 raft_consensus.cc:729] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c660a9fb7174be199d18350f472d637, State: Initialized, Role: FOLLOWER
I20260806 21:06:29.119695 30828 consensus_queue.cc:260] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:29.119771 30829 tablet_bootstrap.cc:654] T 1425e2f04b36418aab14b521e6547f70 P a85bbe87cbd64819965452624929cdf5: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:29.119993 30828 ts_tablet_manager.cc:1428] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 21:06:29.121059 30829 tablet_bootstrap.cc:492] T 1425e2f04b36418aab14b521e6547f70 P a85bbe87cbd64819965452624929cdf5: No bootstrap required, opened a new log
I20260806 21:06:29.121168 30829 ts_tablet_manager.cc:1397] T 1425e2f04b36418aab14b521e6547f70 P a85bbe87cbd64819965452624929cdf5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 21:06:29.121625 30829 raft_consensus.cc:348] T 1425e2f04b36418aab14b521e6547f70 P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:29.121752 30829 raft_consensus.cc:374] T 1425e2f04b36418aab14b521e6547f70 P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:29.121791 30829 raft_consensus.cc:729] T 1425e2f04b36418aab14b521e6547f70 P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a85bbe87cbd64819965452624929cdf5, State: Initialized, Role: FOLLOWER
I20260806 21:06:29.121945 30829 consensus_queue.cc:260] T 1425e2f04b36418aab14b521e6547f70 P a85bbe87cbd64819965452624929cdf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:29.122308 30829 ts_tablet_manager.cc:1428] T 1425e2f04b36418aab14b521e6547f70 P a85bbe87cbd64819965452624929cdf5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:06:29.313305 30865 raft_consensus.cc:482] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:06:29.313481 30865 raft_consensus.cc:504] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:29.314049 30865 leader_election.cc:288] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 754e07fa5de64d52abb908bfa66dc766 (127.27.225.194:39567), a85bbe87cbd64819965452624929cdf5 (127.27.225.195:42839)
I20260806 21:06:29.314502 30612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1425e2f04b36418aab14b521e6547f70" candidate_uuid: "4c660a9fb7174be199d18350f472d637" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "754e07fa5de64d52abb908bfa66dc766" is_pre_election: true
I20260806 21:06:29.314584 30756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1425e2f04b36418aab14b521e6547f70" candidate_uuid: "4c660a9fb7174be199d18350f472d637" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a85bbe87cbd64819965452624929cdf5" is_pre_election: true
I20260806 21:06:29.314673 30612 raft_consensus.cc:2454] T 1425e2f04b36418aab14b521e6547f70 P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c660a9fb7174be199d18350f472d637 in term 0.
I20260806 21:06:29.314715 30756 raft_consensus.cc:2454] T 1425e2f04b36418aab14b521e6547f70 P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c660a9fb7174be199d18350f472d637 in term 0.
I20260806 21:06:29.315078 30401 leader_election.cc:302] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4c660a9fb7174be199d18350f472d637, 754e07fa5de64d52abb908bfa66dc766, a85bbe87cbd64819965452624929cdf5; no voters:
I20260806 21:06:29.315344 30865 raft_consensus.cc:2781] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:06:29.315407 30865 raft_consensus.cc:482] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:06:29.315465 30865 raft_consensus.cc:3037] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:29.316633 30865 raft_consensus.cc:504] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:29.317056 30865 leader_election.cc:288] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637 [CANDIDATE]: Term 1 election: Requested vote from peers 754e07fa5de64d52abb908bfa66dc766 (127.27.225.194:39567), a85bbe87cbd64819965452624929cdf5 (127.27.225.195:42839)
I20260806 21:06:29.317505 30612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1425e2f04b36418aab14b521e6547f70" candidate_uuid: "4c660a9fb7174be199d18350f472d637" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "754e07fa5de64d52abb908bfa66dc766"
I20260806 21:06:29.317513 30756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1425e2f04b36418aab14b521e6547f70" candidate_uuid: "4c660a9fb7174be199d18350f472d637" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a85bbe87cbd64819965452624929cdf5"
I20260806 21:06:29.317653 30612 raft_consensus.cc:3037] T 1425e2f04b36418aab14b521e6547f70 P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:29.317677 30756 raft_consensus.cc:3037] T 1425e2f04b36418aab14b521e6547f70 P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:29.318858 30612 raft_consensus.cc:2454] T 1425e2f04b36418aab14b521e6547f70 P 754e07fa5de64d52abb908bfa66dc766 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c660a9fb7174be199d18350f472d637 in term 1.
I20260806 21:06:29.318917 30756 raft_consensus.cc:2454] T 1425e2f04b36418aab14b521e6547f70 P a85bbe87cbd64819965452624929cdf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c660a9fb7174be199d18350f472d637 in term 1.
I20260806 21:06:29.319252 30399 leader_election.cc:302] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c660a9fb7174be199d18350f472d637, 754e07fa5de64d52abb908bfa66dc766; no voters:
I20260806 21:06:29.319535 30865 raft_consensus.cc:2781] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:29.319671 30865 raft_consensus.cc:686] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637 [term 1 LEADER]: Becoming Leader. State: Replica: 4c660a9fb7174be199d18350f472d637, State: Running, Role: LEADER
I20260806 21:06:29.319867 30865 consensus_queue.cc:237] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:29.322013 30310 catalog_manager.cc:5495] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c660a9fb7174be199d18350f472d637 (127.27.225.193). New cstate: current_term: 1 leader_uuid: "4c660a9fb7174be199d18350f472d637" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } health_report { overall_health: UNKNOWN } } }
I20260806 21:06:29.376602 30756 raft_consensus.cc:1261] T 1425e2f04b36418aab14b521e6547f70 P a85bbe87cbd64819965452624929cdf5 [term 1 FOLLOWER]: Refusing update from remote peer 4c660a9fb7174be199d18350f472d637: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:06:29.376600 30612 raft_consensus.cc:1261] T 1425e2f04b36418aab14b521e6547f70 P 754e07fa5de64d52abb908bfa66dc766 [term 1 FOLLOWER]: Refusing update from remote peer 4c660a9fb7174be199d18350f472d637: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:06:29.377054 30839 consensus_queue.cc:1035] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637 [LEADER]: Connected to new peer: Peer: permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:06:29.377234 30833 consensus_queue.cc:1035] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 21:06:29.392508 30856 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 21:06:30.963405 30307 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:06:30.963596 30307 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:06:30.968048 30307 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de: Sending DeleteTablet for 3 replicas of tablet 1425e2f04b36418aab14b521e6547f70
I20260806 21:06:30.968910 30451 tablet_service.cc:1514] Processing DeleteTablet for tablet 1425e2f04b36418aab14b521e6547f70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:30 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:48848
I20260806 21:06:30.968968 30593 tablet_service.cc:1514] Processing DeleteTablet for tablet 1425e2f04b36418aab14b521e6547f70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:30 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:50090
I20260806 21:06:30.969211 30732 tablet_service.cc:1514] Processing DeleteTablet for tablet 1425e2f04b36418aab14b521e6547f70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:30 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:44740
I20260806 21:06:30.969348 30902 tablet_replica.cc:330] T 1425e2f04b36418aab14b521e6547f70 P 754e07fa5de64d52abb908bfa66dc766: stopping tablet replica
I20260806 21:06:30.969497 30901 tablet_replica.cc:330] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637: stopping tablet replica
I20260806 21:06:30.969496 30902 raft_consensus.cc:2229] T 1425e2f04b36418aab14b521e6547f70 P 754e07fa5de64d52abb908bfa66dc766 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:06:30.969612 30903 tablet_replica.cc:330] T 1425e2f04b36418aab14b521e6547f70 P a85bbe87cbd64819965452624929cdf5: stopping tablet replica
I20260806 21:06:30.970022 30902 raft_consensus.cc:2258] T 1425e2f04b36418aab14b521e6547f70 P 754e07fa5de64d52abb908bfa66dc766 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:30.970186 30901 raft_consensus.cc:2229] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:06:30.970432 30903 raft_consensus.cc:2229] T 1425e2f04b36418aab14b521e6547f70 P a85bbe87cbd64819965452624929cdf5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:06:30.970608 30903 raft_consensus.cc:2258] T 1425e2f04b36418aab14b521e6547f70 P a85bbe87cbd64819965452624929cdf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:30.970937 30901 raft_consensus.cc:2258] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:30.972082 30902 ts_tablet_manager.cc:1905] T 1425e2f04b36418aab14b521e6547f70 P 754e07fa5de64d52abb908bfa66dc766: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:30.972787 30901 ts_tablet_manager.cc:1905] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:30.976174 30902 ts_tablet_manager.cc:1918] T 1425e2f04b36418aab14b521e6547f70 P 754e07fa5de64d52abb908bfa66dc766: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 21:06:30.976307 30902 log.cc:1196] T 1425e2f04b36418aab14b521e6547f70 P 754e07fa5de64d52abb908bfa66dc766: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-1/wal/wals/1425e2f04b36418aab14b521e6547f70
I20260806 21:06:30.976639 30903 ts_tablet_manager.cc:1905] T 1425e2f04b36418aab14b521e6547f70 P a85bbe87cbd64819965452624929cdf5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:30.976795 30902 ts_tablet_manager.cc:1939] T 1425e2f04b36418aab14b521e6547f70 P 754e07fa5de64d52abb908bfa66dc766: Deleting consensus metadata
I20260806 21:06:30.977092 30901 ts_tablet_manager.cc:1918] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 21:06:30.977237 30901 log.cc:1196] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-0/wal/wals/1425e2f04b36418aab14b521e6547f70
I20260806 21:06:30.977706 30901 ts_tablet_manager.cc:1939] T 1425e2f04b36418aab14b521e6547f70 P 4c660a9fb7174be199d18350f472d637: Deleting consensus metadata
I20260806 21:06:30.977993 30295 catalog_manager.cc:4841] TS 754e07fa5de64d52abb908bfa66dc766 (127.27.225.194:39567): tablet 1425e2f04b36418aab14b521e6547f70 (table test-table [id=292af2dd8fd5456983a70bb05e64b5cb]) successfully deleted
I20260806 21:06:30.978704 30295 catalog_manager.cc:4841] TS 4c660a9fb7174be199d18350f472d637 (127.27.225.193:40909): tablet 1425e2f04b36418aab14b521e6547f70 (table test-table [id=292af2dd8fd5456983a70bb05e64b5cb]) successfully deleted
I20260806 21:06:30.980436 30903 ts_tablet_manager.cc:1918] T 1425e2f04b36418aab14b521e6547f70 P a85bbe87cbd64819965452624929cdf5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 21:06:30.980518 30903 log.cc:1196] T 1425e2f04b36418aab14b521e6547f70 P a85bbe87cbd64819965452624929cdf5: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/1425e2f04b36418aab14b521e6547f70
I20260806 21:06:30.980820 30903 ts_tablet_manager.cc:1939] T 1425e2f04b36418aab14b521e6547f70 P a85bbe87cbd64819965452624929cdf5: Deleting consensus metadata
I20260806 21:06:30.982214 30297 catalog_manager.cc:4841] TS a85bbe87cbd64819965452624929cdf5 (127.27.225.195:42839): tablet 1425e2f04b36418aab14b521e6547f70 (table test-table [id=292af2dd8fd5456983a70bb05e64b5cb]) successfully deleted
Aug 06 21:06:30 dist-test-slave-69rg krb5kdc[30263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050390, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 21:06:30 dist-test-slave-69rg krb5kdc[30263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050390, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:31.007907 30911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:30.995851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47948 (local address 127.27.225.254:38963)
0806 21:06:30.996188 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:30.996196 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:30.996214 (+ 18us) server_negotiation.cc:407] Connection header received
0806 21:06:30.996345 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:30.996348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:30.996403 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:30.996517 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:30.997330 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:30.997983 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:30.998922 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:30.999159 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:30.999251 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:31.002376 (+ 3125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:31.002380 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:31.002382 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:31.002415 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:31.004478 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:31.005264 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:31.005267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:31.005268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:31.005481 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:31.006528 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:31.006531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:31.006533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:31.006887 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:31.007037 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:31.007543 (+ 506us) server_negotiation.cc:299] Negotiation successful
0806 21:06:31.007727 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":156,"threads_started":1}
I20260806 21:06:31.012509 30307 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47948:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:06:31.013571 30307 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:06:31.021008 30732 tablet_service.cc:1467] Processing CreateTablet for tablet 85680a310b714ae78a0288616982563c (DEFAULT_TABLE table=test-table [id=60915556490a4ff99a3c4bd901b1e033]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:31.021008 30593 tablet_service.cc:1467] Processing CreateTablet for tablet 85680a310b714ae78a0288616982563c (DEFAULT_TABLE table=test-table [id=60915556490a4ff99a3c4bd901b1e033]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:31.021008 30451 tablet_service.cc:1467] Processing CreateTablet for tablet 85680a310b714ae78a0288616982563c (DEFAULT_TABLE table=test-table [id=60915556490a4ff99a3c4bd901b1e033]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:31.021377 30593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85680a310b714ae78a0288616982563c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:31.021377 30732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85680a310b714ae78a0288616982563c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:31.021390 30451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85680a310b714ae78a0288616982563c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:31.023772 30912 tablet_bootstrap.cc:492] T 85680a310b714ae78a0288616982563c P 4c660a9fb7174be199d18350f472d637: Bootstrap starting.
I20260806 21:06:31.024256 30913 tablet_bootstrap.cc:492] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5: Bootstrap starting.
I20260806 21:06:31.024339 30914 tablet_bootstrap.cc:492] T 85680a310b714ae78a0288616982563c P 754e07fa5de64d52abb908bfa66dc766: Bootstrap starting.
I20260806 21:06:31.025087 30912 tablet_bootstrap.cc:654] T 85680a310b714ae78a0288616982563c P 4c660a9fb7174be199d18350f472d637: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:31.025302 30914 tablet_bootstrap.cc:654] T 85680a310b714ae78a0288616982563c P 754e07fa5de64d52abb908bfa66dc766: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:31.025301 30913 tablet_bootstrap.cc:654] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:31.026862 30912 tablet_bootstrap.cc:492] T 85680a310b714ae78a0288616982563c P 4c660a9fb7174be199d18350f472d637: No bootstrap required, opened a new log
I20260806 21:06:31.026906 30913 tablet_bootstrap.cc:492] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5: No bootstrap required, opened a new log
I20260806 21:06:31.026943 30912 ts_tablet_manager.cc:1397] T 85680a310b714ae78a0288616982563c P 4c660a9fb7174be199d18350f472d637: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:06:31.027037 30914 tablet_bootstrap.cc:492] T 85680a310b714ae78a0288616982563c P 754e07fa5de64d52abb908bfa66dc766: No bootstrap required, opened a new log
I20260806 21:06:31.027029 30913 ts_tablet_manager.cc:1397] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:06:31.027180 30914 ts_tablet_manager.cc:1397] T 85680a310b714ae78a0288616982563c P 754e07fa5de64d52abb908bfa66dc766: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 21:06:31.027436 30912 raft_consensus.cc:348] T 85680a310b714ae78a0288616982563c P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:31.027573 30912 raft_consensus.cc:374] T 85680a310b714ae78a0288616982563c P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:31.027611 30912 raft_consensus.cc:729] T 85680a310b714ae78a0288616982563c P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c660a9fb7174be199d18350f472d637, State: Initialized, Role: FOLLOWER
I20260806 21:06:31.027647 30913 raft_consensus.cc:348] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:31.027776 30913 raft_consensus.cc:374] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:31.027715 30914 raft_consensus.cc:348] T 85680a310b714ae78a0288616982563c P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:31.027755 30912 consensus_queue.cc:260] T 85680a310b714ae78a0288616982563c P 4c660a9fb7174be199d18350f472d637 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:31.027812 30913 raft_consensus.cc:729] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a85bbe87cbd64819965452624929cdf5, State: Initialized, Role: FOLLOWER
I20260806 21:06:31.027823 30914 raft_consensus.cc:374] T 85680a310b714ae78a0288616982563c P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:31.027868 30914 raft_consensus.cc:729] T 85680a310b714ae78a0288616982563c P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 754e07fa5de64d52abb908bfa66dc766, State: Initialized, Role: FOLLOWER
I20260806 21:06:31.027959 30913 consensus_queue.cc:260] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:31.028047 30912 ts_tablet_manager.cc:1428] T 85680a310b714ae78a0288616982563c P 4c660a9fb7174be199d18350f472d637: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 21:06:31.028030 30914 consensus_queue.cc:260] T 85680a310b714ae78a0288616982563c P 754e07fa5de64d52abb908bfa66dc766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:31.028411 30914 ts_tablet_manager.cc:1428] T 85680a310b714ae78a0288616982563c P 754e07fa5de64d52abb908bfa66dc766: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 21:06:31.028410 30913 ts_tablet_manager.cc:1428] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:06:31.112841 30919 raft_consensus.cc:482] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:06:31.113040 30919 raft_consensus.cc:504] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:31.114140 30919 leader_election.cc:288] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c660a9fb7174be199d18350f472d637 (127.27.225.193:40909), 754e07fa5de64d52abb908bfa66dc766 (127.27.225.194:39567)
I20260806 21:06:31.118373 30922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:31.114593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:48165 (local address 127.27.225.193:40909)
0806 21:06:31.114889 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:31.114896 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:31.114910 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:06:31.114960 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:31.114962 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:31.115010 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:31.115125 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:31.115436 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.116061 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.117266 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.117704 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.117880 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:31.117937 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:31.118145 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 21:06:31.118229 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":128,"threads_started":1}
I20260806 21:06:31.118386 30921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:31.114412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:40909 (local address 127.27.225.195:48165)
0806 21:06:31.114712 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:31.114729 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:31.114837 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:31.115141 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:31.115144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:31.115163 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:31.115287 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:31.115298 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.116187 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.116190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:31.117127 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:31.117136 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.117887 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.117890 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:31.117986 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:31.118008 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 21:06:31.118129 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 21:06:31.118209 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":138,"threads_started":1}
I20260806 21:06:31.118676 30923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:31.114845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:39567 (local address 127.27.225.195:38831)
0806 21:06:31.115143 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:31.115159 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:31.115257 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:31.115721 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:31.115724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:31.115740 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:31.115898 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:31.115904 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.116747 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.116751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:31.117486 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:31.117494 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.118211 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.118215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:31.118319 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:31.118348 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 21:06:31.118438 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 21:06:31.118528 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":120,"threads_started":1}
I20260806 21:06:31.118863 30471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85680a310b714ae78a0288616982563c" candidate_uuid: "a85bbe87cbd64819965452624929cdf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c660a9fb7174be199d18350f472d637" is_pre_election: true
I20260806 21:06:31.118955 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:31.115264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:38831 (local address 127.27.225.194:39567)
0806 21:06:31.115514 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:31.115521 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:31.115532 (+ 11us) server_negotiation.cc:407] Connection header received
0806 21:06:31.115574 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:31.115576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:31.115613 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:31.115712 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:31.116013 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.116618 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.117614 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.118092 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.118221 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:31.118273 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:31.118769 (+ 496us) server_negotiation.cc:299] Negotiation successful
0806 21:06:31.118859 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":109,"threads_started":1}
I20260806 21:06:31.119019 30471 raft_consensus.cc:2454] T 85680a310b714ae78a0288616982563c P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a85bbe87cbd64819965452624929cdf5 in term 0.
I20260806 21:06:31.119513 30687 leader_election.cc:302] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c660a9fb7174be199d18350f472d637, a85bbe87cbd64819965452624929cdf5; no voters:
I20260806 21:06:31.119784 30919 raft_consensus.cc:2781] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:06:31.119860 30919 raft_consensus.cc:482] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:06:31.119899 30919 raft_consensus.cc:3037] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:31.120081 30613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85680a310b714ae78a0288616982563c" candidate_uuid: "a85bbe87cbd64819965452624929cdf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "754e07fa5de64d52abb908bfa66dc766" is_pre_election: true
I20260806 21:06:31.120245 30613 raft_consensus.cc:2454] T 85680a310b714ae78a0288616982563c P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a85bbe87cbd64819965452624929cdf5 in term 0.
I20260806 21:06:31.121090 30919 raft_consensus.cc:504] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:31.121459 30919 leader_election.cc:288] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5 [CANDIDATE]: Term 1 election: Requested vote from peers 4c660a9fb7174be199d18350f472d637 (127.27.225.193:40909), 754e07fa5de64d52abb908bfa66dc766 (127.27.225.194:39567)
I20260806 21:06:31.121975 30613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85680a310b714ae78a0288616982563c" candidate_uuid: "a85bbe87cbd64819965452624929cdf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "754e07fa5de64d52abb908bfa66dc766"
I20260806 21:06:31.122013 30471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85680a310b714ae78a0288616982563c" candidate_uuid: "a85bbe87cbd64819965452624929cdf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c660a9fb7174be199d18350f472d637"
I20260806 21:06:31.122125 30613 raft_consensus.cc:3037] T 85680a310b714ae78a0288616982563c P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:31.122140 30471 raft_consensus.cc:3037] T 85680a310b714ae78a0288616982563c P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:31.123462 30613 raft_consensus.cc:2454] T 85680a310b714ae78a0288616982563c P 754e07fa5de64d52abb908bfa66dc766 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a85bbe87cbd64819965452624929cdf5 in term 1.
I20260806 21:06:31.123462 30471 raft_consensus.cc:2454] T 85680a310b714ae78a0288616982563c P 4c660a9fb7174be199d18350f472d637 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a85bbe87cbd64819965452624929cdf5 in term 1.
I20260806 21:06:31.123780 30687 leader_election.cc:302] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c660a9fb7174be199d18350f472d637, a85bbe87cbd64819965452624929cdf5; no voters:
I20260806 21:06:31.123975 30919 raft_consensus.cc:2781] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:31.124222 30919 raft_consensus.cc:686] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5 [term 1 LEADER]: Becoming Leader. State: Replica: a85bbe87cbd64819965452624929cdf5, State: Running, Role: LEADER
I20260806 21:06:31.124516 30919 consensus_queue.cc:237] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:31.127602 30311 catalog_manager.cc:5495] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5 reported cstate change: term changed from 0 to 1, leader changed from <none> to a85bbe87cbd64819965452624929cdf5 (127.27.225.195). New cstate: current_term: 1 leader_uuid: "a85bbe87cbd64819965452624929cdf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } health_report { overall_health: HEALTHY } } }
I20260806 21:06:31.166352 30929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:31.161954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55300 (local address 127.27.225.195:42839)
0806 21:06:31.162284 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:31.162333 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:31.162345 (+ 12us) server_negotiation.cc:407] Connection header received
0806 21:06:31.162419 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:31.162422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:31.162477 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:31.162623 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:06:31.162930 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.163605 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.164621 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.164905 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.164994 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:31.165316 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:06:31.165853 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:06:31.165951 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:31.166086 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 21:06:31.166146 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":132,"threads_started":1}
I20260806 21:06:31.177306 30932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:31.174023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:40909 (local address 127.27.225.195:54187)
0806 21:06:31.174337 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:31.174350 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:31.174444 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:31.174649 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:31.174651 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:31.174657 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 21:06:31.174765 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:31.174769 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.175468 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.175470 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:31.176121 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:31.176129 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.176593 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.176596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:31.176664 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:31.176708 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 21:06:31.177047 (+ 339us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 21:06:31.177052 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 21:06:31.177101 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 21:06:31.177169 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":125,"threads_started":1}
I20260806 21:06:31.177297 30922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:31.174145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:54187 (local address 127.27.225.193:40909)
0806 21:06:31.174291 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:31.174295 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:31.174417 (+ 122us) server_negotiation.cc:407] Connection header received
0806 21:06:31.174515 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:31.174517 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:31.174560 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:31.174627 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:06:31.174881 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.175381 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.176244 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.176479 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.176568 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:31.176868 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:06:31.176955 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:06:31.177033 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:31.177117 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 21:06:31.177175 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 21:06:31.184018 30613 raft_consensus.cc:1261] T 85680a310b714ae78a0288616982563c P 754e07fa5de64d52abb908bfa66dc766 [term 1 FOLLOWER]: Refusing update from remote peer a85bbe87cbd64819965452624929cdf5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:06:31.184304 30471 raft_consensus.cc:1261] T 85680a310b714ae78a0288616982563c P 4c660a9fb7174be199d18350f472d637 [term 1 FOLLOWER]: Refusing update from remote peer a85bbe87cbd64819965452624929cdf5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:06:31.184569 30919 consensus_queue.cc:1035] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:06:31.184857 30925 consensus_queue.cc:1035] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:06:31.203754 30929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:31.200358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:59307 (local address 127.27.225.195:42839)
0806 21:06:31.200454 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:31.200458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:31.200468 (+ 10us) server_negotiation.cc:407] Connection header received
0806 21:06:31.200507 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:31.200509 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:31.200545 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:31.200636 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:06:31.200855 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.201425 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.202338 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.202577 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.202685 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:31.203129 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:06:31.203249 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:06:31.203337 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:31.203522 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 21:06:31.203589 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 21:06:31.204211 30936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:31.199985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:42839 (local address 127.27.225.193:59307)
0806 21:06:31.200285 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:31.200304 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:31.200420 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:31.200647 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:31.200650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:31.200658 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 21:06:31.200762 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:31.200768 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.201530 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.201534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:31.202214 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:31.202223 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.202686 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.202690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:31.202782 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:31.202819 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 21:06:31.203388 (+ 569us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 21:06:31.203400 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 21:06:31.203915 (+ 515us) client_negotiation.cc:240] Negotiation successful
0806 21:06:31.204036 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":100,"threads_started":1}
I20260806 21:06:31.227360 30310 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:06:31.227568 30310 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:06:31.230168 30310 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de: Sending DeleteTablet for 3 replicas of tablet 85680a310b714ae78a0288616982563c
I20260806 21:06:31.230890 30451 tablet_service.cc:1514] Processing DeleteTablet for tablet 85680a310b714ae78a0288616982563c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:31 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:48848
I20260806 21:06:31.231017 30593 tablet_service.cc:1514] Processing DeleteTablet for tablet 85680a310b714ae78a0288616982563c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:31 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:50090
I20260806 21:06:31.231125 30901 tablet_replica.cc:330] T 85680a310b714ae78a0288616982563c P 4c660a9fb7174be199d18350f472d637: stopping tablet replica
I20260806 21:06:31.231248 30901 raft_consensus.cc:2229] T 85680a310b714ae78a0288616982563c P 4c660a9fb7174be199d18350f472d637 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:06:31.231405 30901 raft_consensus.cc:2258] T 85680a310b714ae78a0288616982563c P 4c660a9fb7174be199d18350f472d637 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:31.231431 30732 tablet_service.cc:1514] Processing DeleteTablet for tablet 85680a310b714ae78a0288616982563c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:31 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:44740
I20260806 21:06:31.231614 30902 tablet_replica.cc:330] T 85680a310b714ae78a0288616982563c P 754e07fa5de64d52abb908bfa66dc766: stopping tablet replica
I20260806 21:06:31.231633 30903 tablet_replica.cc:330] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5: stopping tablet replica
I20260806 21:06:31.231742 30903 raft_consensus.cc:2229] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:06:31.231751 30902 raft_consensus.cc:2229] T 85680a310b714ae78a0288616982563c P 754e07fa5de64d52abb908bfa66dc766 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:06:31.231886 30902 raft_consensus.cc:2258] T 85680a310b714ae78a0288616982563c P 754e07fa5de64d52abb908bfa66dc766 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:31.232117 30903 raft_consensus.cc:2258] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:31.232710 30902 ts_tablet_manager.cc:1905] T 85680a310b714ae78a0288616982563c P 754e07fa5de64d52abb908bfa66dc766: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:31.232710 30901 ts_tablet_manager.cc:1905] T 85680a310b714ae78a0288616982563c P 4c660a9fb7174be199d18350f472d637: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:31.232995 30903 ts_tablet_manager.cc:1905] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:31.235388 30902 ts_tablet_manager.cc:1918] T 85680a310b714ae78a0288616982563c P 754e07fa5de64d52abb908bfa66dc766: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:06:31.235388 30901 ts_tablet_manager.cc:1918] T 85680a310b714ae78a0288616982563c P 4c660a9fb7174be199d18350f472d637: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:06:31.235471 30901 log.cc:1196] T 85680a310b714ae78a0288616982563c P 4c660a9fb7174be199d18350f472d637: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-0/wal/wals/85680a310b714ae78a0288616982563c
I20260806 21:06:31.235471 30902 log.cc:1196] T 85680a310b714ae78a0288616982563c P 754e07fa5de64d52abb908bfa66dc766: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-1/wal/wals/85680a310b714ae78a0288616982563c
I20260806 21:06:31.235734 30902 ts_tablet_manager.cc:1939] T 85680a310b714ae78a0288616982563c P 754e07fa5de64d52abb908bfa66dc766: Deleting consensus metadata
I20260806 21:06:31.235734 30901 ts_tablet_manager.cc:1939] T 85680a310b714ae78a0288616982563c P 4c660a9fb7174be199d18350f472d637: Deleting consensus metadata
I20260806 21:06:31.235816 30903 ts_tablet_manager.cc:1918] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:06:31.235893 30903 log.cc:1196] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/85680a310b714ae78a0288616982563c
I20260806 21:06:31.236194 30903 ts_tablet_manager.cc:1939] T 85680a310b714ae78a0288616982563c P a85bbe87cbd64819965452624929cdf5: Deleting consensus metadata
I20260806 21:06:31.236959 30295 catalog_manager.cc:4841] TS 754e07fa5de64d52abb908bfa66dc766 (127.27.225.194:39567): tablet 85680a310b714ae78a0288616982563c (table test-table [id=60915556490a4ff99a3c4bd901b1e033]) successfully deleted
I20260806 21:06:31.236898 30310 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47948:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:06:31.237373 30310 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:06:31.237950 30297 catalog_manager.cc:4841] TS a85bbe87cbd64819965452624929cdf5 (127.27.225.195:42839): tablet 85680a310b714ae78a0288616982563c (table test-table [id=60915556490a4ff99a3c4bd901b1e033]) successfully deleted
I20260806 21:06:31.238253 30295 catalog_manager.cc:4841] TS 4c660a9fb7174be199d18350f472d637 (127.27.225.193:40909): tablet 85680a310b714ae78a0288616982563c (table test-table [id=60915556490a4ff99a3c4bd901b1e033]) successfully deleted
I20260806 21:06:31.242460 30451 tablet_service.cc:1467] Processing CreateTablet for tablet df16f34990c6480d93e8fe4aaec8bf17 (DEFAULT_TABLE table=test-table [id=ae3c2eb331e04b63b7a74740ece21585]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:31.242552 30593 tablet_service.cc:1467] Processing CreateTablet for tablet df16f34990c6480d93e8fe4aaec8bf17 (DEFAULT_TABLE table=test-table [id=ae3c2eb331e04b63b7a74740ece21585]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:31.242803 30451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df16f34990c6480d93e8fe4aaec8bf17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:31.242861 30593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df16f34990c6480d93e8fe4aaec8bf17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:31.244807 30912 tablet_bootstrap.cc:492] T df16f34990c6480d93e8fe4aaec8bf17 P 4c660a9fb7174be199d18350f472d637: Bootstrap starting.
I20260806 21:06:31.245450 30732 tablet_service.cc:1467] Processing CreateTablet for tablet df16f34990c6480d93e8fe4aaec8bf17 (DEFAULT_TABLE table=test-table [id=ae3c2eb331e04b63b7a74740ece21585]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:31.245604 30912 tablet_bootstrap.cc:654] T df16f34990c6480d93e8fe4aaec8bf17 P 4c660a9fb7174be199d18350f472d637: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:31.245718 30732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df16f34990c6480d93e8fe4aaec8bf17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:31.246699 30912 tablet_bootstrap.cc:492] T df16f34990c6480d93e8fe4aaec8bf17 P 4c660a9fb7174be199d18350f472d637: No bootstrap required, opened a new log
I20260806 21:06:31.246802 30912 ts_tablet_manager.cc:1397] T df16f34990c6480d93e8fe4aaec8bf17 P 4c660a9fb7174be199d18350f472d637: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 21:06:31.247198 30912 raft_consensus.cc:348] T df16f34990c6480d93e8fe4aaec8bf17 P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:31.247339 30912 raft_consensus.cc:374] T df16f34990c6480d93e8fe4aaec8bf17 P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:31.247380 30912 raft_consensus.cc:729] T df16f34990c6480d93e8fe4aaec8bf17 P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c660a9fb7174be199d18350f472d637, State: Initialized, Role: FOLLOWER
I20260806 21:06:31.247499 30912 consensus_queue.cc:260] T df16f34990c6480d93e8fe4aaec8bf17 P 4c660a9fb7174be199d18350f472d637 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:31.247785 30912 ts_tablet_manager.cc:1428] T df16f34990c6480d93e8fe4aaec8bf17 P 4c660a9fb7174be199d18350f472d637: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:06:31.248564 30914 tablet_bootstrap.cc:492] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766: Bootstrap starting.
I20260806 21:06:31.249607 30914 tablet_bootstrap.cc:654] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:31.250326 30913 tablet_bootstrap.cc:492] T df16f34990c6480d93e8fe4aaec8bf17 P a85bbe87cbd64819965452624929cdf5: Bootstrap starting.
I20260806 21:06:31.251361 30913 tablet_bootstrap.cc:654] T df16f34990c6480d93e8fe4aaec8bf17 P a85bbe87cbd64819965452624929cdf5: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:31.252707 30914 tablet_bootstrap.cc:492] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766: No bootstrap required, opened a new log
I20260806 21:06:31.252738 30913 tablet_bootstrap.cc:492] T df16f34990c6480d93e8fe4aaec8bf17 P a85bbe87cbd64819965452624929cdf5: No bootstrap required, opened a new log
I20260806 21:06:31.252806 30914 ts_tablet_manager.cc:1397] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260806 21:06:31.252857 30913 ts_tablet_manager.cc:1397] T df16f34990c6480d93e8fe4aaec8bf17 P a85bbe87cbd64819965452624929cdf5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:06:31.253273 30914 raft_consensus.cc:348] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:31.253352 30913 raft_consensus.cc:348] T df16f34990c6480d93e8fe4aaec8bf17 P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:31.253451 30914 raft_consensus.cc:374] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:31.253469 30913 raft_consensus.cc:374] T df16f34990c6480d93e8fe4aaec8bf17 P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:31.253487 30914 raft_consensus.cc:729] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 754e07fa5de64d52abb908bfa66dc766, State: Initialized, Role: FOLLOWER
I20260806 21:06:31.253506 30913 raft_consensus.cc:729] T df16f34990c6480d93e8fe4aaec8bf17 P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a85bbe87cbd64819965452624929cdf5, State: Initialized, Role: FOLLOWER
I20260806 21:06:31.253619 30914 consensus_queue.cc:260] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:31.253644 30913 consensus_queue.cc:260] T df16f34990c6480d93e8fe4aaec8bf17 P a85bbe87cbd64819965452624929cdf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:31.253927 30914 ts_tablet_manager.cc:1428] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 21:06:31.254346 30913 ts_tablet_manager.cc:1428] T df16f34990c6480d93e8fe4aaec8bf17 P a85bbe87cbd64819965452624929cdf5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:06:31.341620 30920 raft_consensus.cc:482] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:06:31.341830 30920 raft_consensus.cc:504] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:31.342404 30920 leader_election.cc:288] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c660a9fb7174be199d18350f472d637 (127.27.225.193:40909), a85bbe87cbd64819965452624929cdf5 (127.27.225.195:42839)
I20260806 21:06:31.342931 30756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df16f34990c6480d93e8fe4aaec8bf17" candidate_uuid: "754e07fa5de64d52abb908bfa66dc766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a85bbe87cbd64819965452624929cdf5" is_pre_election: true
I20260806 21:06:31.342917 30471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df16f34990c6480d93e8fe4aaec8bf17" candidate_uuid: "754e07fa5de64d52abb908bfa66dc766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c660a9fb7174be199d18350f472d637" is_pre_election: true
I20260806 21:06:31.343084 30471 raft_consensus.cc:2454] T df16f34990c6480d93e8fe4aaec8bf17 P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 754e07fa5de64d52abb908bfa66dc766 in term 0.
I20260806 21:06:31.343103 30756 raft_consensus.cc:2454] T df16f34990c6480d93e8fe4aaec8bf17 P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 754e07fa5de64d52abb908bfa66dc766 in term 0.
I20260806 21:06:31.343472 30546 leader_election.cc:302] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 754e07fa5de64d52abb908bfa66dc766, a85bbe87cbd64819965452624929cdf5; no voters:
I20260806 21:06:31.343674 30920 raft_consensus.cc:2781] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:06:31.343751 30920 raft_consensus.cc:482] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:06:31.343806 30920 raft_consensus.cc:3037] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:31.344875 30920 raft_consensus.cc:504] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:31.345254 30920 leader_election.cc:288] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766 [CANDIDATE]: Term 1 election: Requested vote from peers 4c660a9fb7174be199d18350f472d637 (127.27.225.193:40909), a85bbe87cbd64819965452624929cdf5 (127.27.225.195:42839)
I20260806 21:06:31.345635 30756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df16f34990c6480d93e8fe4aaec8bf17" candidate_uuid: "754e07fa5de64d52abb908bfa66dc766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a85bbe87cbd64819965452624929cdf5"
I20260806 21:06:31.345638 30471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df16f34990c6480d93e8fe4aaec8bf17" candidate_uuid: "754e07fa5de64d52abb908bfa66dc766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c660a9fb7174be199d18350f472d637"
I20260806 21:06:31.345777 30756 raft_consensus.cc:3037] T df16f34990c6480d93e8fe4aaec8bf17 P a85bbe87cbd64819965452624929cdf5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:31.345777 30471 raft_consensus.cc:3037] T df16f34990c6480d93e8fe4aaec8bf17 P 4c660a9fb7174be199d18350f472d637 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:31.347000 30756 raft_consensus.cc:2454] T df16f34990c6480d93e8fe4aaec8bf17 P a85bbe87cbd64819965452624929cdf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 754e07fa5de64d52abb908bfa66dc766 in term 1.
I20260806 21:06:31.347091 30471 raft_consensus.cc:2454] T df16f34990c6480d93e8fe4aaec8bf17 P 4c660a9fb7174be199d18350f472d637 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 754e07fa5de64d52abb908bfa66dc766 in term 1.
I20260806 21:06:31.347421 30546 leader_election.cc:302] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4c660a9fb7174be199d18350f472d637, 754e07fa5de64d52abb908bfa66dc766, a85bbe87cbd64819965452624929cdf5; no voters:
I20260806 21:06:31.347695 30920 raft_consensus.cc:2781] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:31.347884 30920 raft_consensus.cc:686] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766 [term 1 LEADER]: Becoming Leader. State: Replica: 754e07fa5de64d52abb908bfa66dc766, State: Running, Role: LEADER
I20260806 21:06:31.348140 30920 consensus_queue.cc:237] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } }
I20260806 21:06:31.350502 30310 catalog_manager.cc:5495] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766 reported cstate change: term changed from 0 to 1, leader changed from <none> to 754e07fa5de64d52abb908bfa66dc766 (127.27.225.194). New cstate: current_term: 1 leader_uuid: "754e07fa5de64d52abb908bfa66dc766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "754e07fa5de64d52abb908bfa66dc766" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 39567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 } health_report { overall_health: UNKNOWN } } }
I20260806 21:06:31.386210 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:31.382181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44228 (local address 127.27.225.194:39567)
0806 21:06:31.382287 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:31.382294 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:31.382312 (+ 18us) server_negotiation.cc:407] Connection header received
0806 21:06:31.382527 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:31.382530 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:31.382582 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:31.382699 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:06:31.382914 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.383429 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.384297 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.384518 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.384624 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:31.384906 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:06:31.385360 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:06:31.385485 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:31.385555 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 21:06:31.385612 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 21:06:31.397323 30946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:31.393334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:40909 (local address 127.27.225.194:54753)
0806 21:06:31.393640 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:31.393658 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:31.393860 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:31.394142 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:31.394145 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:31.394153 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 21:06:31.394301 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:31.394307 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.395257 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.395262 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:31.396017 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:31.396027 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.396550 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.396554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:31.396657 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:31.396700 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 21:06:31.397010 (+ 310us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 21:06:31.397018 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 21:06:31.397084 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 21:06:31.397167 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":94,"threads_started":1}
I20260806 21:06:31.397302 30922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:31.393595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:54753 (local address 127.27.225.193:40909)
0806 21:06:31.393733 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:31.393740 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:31.393755 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:06:31.393983 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:31.393986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:31.394044 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:31.394120 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:06:31.394432 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.395152 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.396154 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.396437 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.396549 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:31.396849 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:06:31.396927 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:06:31.397006 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:31.397092 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 21:06:31.397143 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 21:06:31.405920 30756 raft_consensus.cc:1261] T df16f34990c6480d93e8fe4aaec8bf17 P a85bbe87cbd64819965452624929cdf5 [term 1 FOLLOWER]: Refusing update from remote peer 754e07fa5de64d52abb908bfa66dc766: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:06:31.405947 30471 raft_consensus.cc:1261] T df16f34990c6480d93e8fe4aaec8bf17 P 4c660a9fb7174be199d18350f472d637 [term 1 FOLLOWER]: Refusing update from remote peer 754e07fa5de64d52abb908bfa66dc766: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:06:31.406390 30942 consensus_queue.cc:1035] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a85bbe87cbd64819965452624929cdf5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 42839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:06:31.406625 30920 consensus_queue.cc:1035] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c660a9fb7174be199d18350f472d637" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:06:31.425328 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:31.421952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:50015 (local address 127.27.225.194:39567)
0806 21:06:31.422062 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:31.422067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:31.422081 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:06:31.422183 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:31.422186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:31.422243 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:31.422330 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:06:31.422570 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.423250 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.424199 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.424520 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.424601 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:31.424879 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:06:31.424969 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:06:31.425037 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:31.425167 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 21:06:31.425217 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 21:06:31.425347 30936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:31.421727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:39567 (local address 127.27.225.193:50015)
0806 21:06:31.422003 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:31.422019 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:31.422099 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:31.422354 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:31.422357 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:31.422364 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 21:06:31.422473 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:31.422479 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.423412 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.423415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:31.424053 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:31.424066 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:31.424634 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:31.424637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:31.424704 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:31.424737 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 21:06:31.425063 (+ 326us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 21:06:31.425069 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 21:06:31.425155 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 21:06:31.425243 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184}
W20260806 21:06:31.426223 30886 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 21:06:32.998654 30311 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:06:32.998816 30311 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:06:33.001754 30311 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2508f2d546dd43549606b08b33b1c4de: Sending DeleteTablet for 3 replicas of tablet df16f34990c6480d93e8fe4aaec8bf17
I20260806 21:06:33.002606 30593 tablet_service.cc:1514] Processing DeleteTablet for tablet df16f34990c6480d93e8fe4aaec8bf17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:32 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:50090
I20260806 21:06:33.002635 30451 tablet_service.cc:1514] Processing DeleteTablet for tablet df16f34990c6480d93e8fe4aaec8bf17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:32 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:48848
I20260806 21:06:33.002972 30732 tablet_service.cc:1514] Processing DeleteTablet for tablet df16f34990c6480d93e8fe4aaec8bf17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:32 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:44740
I20260806 21:06:33.003053 30975 tablet_replica.cc:330] T df16f34990c6480d93e8fe4aaec8bf17 P 4c660a9fb7174be199d18350f472d637: stopping tablet replica
I20260806 21:06:33.003262 30975 raft_consensus.cc:2229] T df16f34990c6480d93e8fe4aaec8bf17 P 4c660a9fb7174be199d18350f472d637 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:06:33.003413 30975 raft_consensus.cc:2258] T df16f34990c6480d93e8fe4aaec8bf17 P 4c660a9fb7174be199d18350f472d637 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:33.003428 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 30383
I20260806 21:06:33.007117 30976 tablet_replica.cc:330] T df16f34990c6480d93e8fe4aaec8bf17 P a85bbe87cbd64819965452624929cdf5: stopping tablet replica
I20260806 21:06:33.007347 30976 raft_consensus.cc:2229] T df16f34990c6480d93e8fe4aaec8bf17 P a85bbe87cbd64819965452624929cdf5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:06:33.007522 30976 raft_consensus.cc:2258] T df16f34990c6480d93e8fe4aaec8bf17 P a85bbe87cbd64819965452624929cdf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:33.007834 30974 tablet_replica.cc:330] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766: stopping tablet replica
I20260806 21:06:33.008009 30974 raft_consensus.cc:2229] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:06:33.008351 30974 raft_consensus.cc:2258] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:33.008761 30976 ts_tablet_manager.cc:1905] T df16f34990c6480d93e8fe4aaec8bf17 P a85bbe87cbd64819965452624929cdf5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:33.009063 30974 ts_tablet_manager.cc:1905] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:33.011461 30976 ts_tablet_manager.cc:1918] T df16f34990c6480d93e8fe4aaec8bf17 P a85bbe87cbd64819965452624929cdf5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 21:06:33.011595 30976 log.cc:1196] T df16f34990c6480d93e8fe4aaec8bf17 P a85bbe87cbd64819965452624929cdf5: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/df16f34990c6480d93e8fe4aaec8bf17
I20260806 21:06:33.011854 30976 ts_tablet_manager.cc:1939] T df16f34990c6480d93e8fe4aaec8bf17 P a85bbe87cbd64819965452624929cdf5: Deleting consensus metadata
I20260806 21:06:33.012686 30974 ts_tablet_manager.cc:1918] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 21:06:33.012795 30974 log.cc:1196] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050378084738-28551-0/minicluster-data/ts-1/wal/wals/df16f34990c6480d93e8fe4aaec8bf17
I20260806 21:06:33.012931 30297 catalog_manager.cc:4841] TS a85bbe87cbd64819965452624929cdf5 (127.27.225.195:42839): tablet df16f34990c6480d93e8fe4aaec8bf17 (table test-table [id=ae3c2eb331e04b63b7a74740ece21585]) successfully deleted
I20260806 21:06:33.013116 30974 ts_tablet_manager.cc:1939] T df16f34990c6480d93e8fe4aaec8bf17 P 754e07fa5de64d52abb908bfa66dc766: Deleting consensus metadata
I20260806 21:06:33.014531 30295 catalog_manager.cc:4841] TS 754e07fa5de64d52abb908bfa66dc766 (127.27.225.194:39567): tablet df16f34990c6480d93e8fe4aaec8bf17 (table test-table [id=ae3c2eb331e04b63b7a74740ece21585]) successfully deleted
W20260806 21:06:33.015087 30295 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.225.193:40909 (error 108)
I20260806 21:06:33.015349 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 30527
I20260806 21:06:33.017611 30978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:33.017086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:40909 (local address 127.0.0.1:59606)
0806 21:06:33.017441 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:33.017516 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.193:40909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":177,"threads_started":1}
W20260806 21:06:33.017912 30295 catalog_manager.cc:4584] TS 4c660a9fb7174be199d18350f472d637 (127.27.225.193:40909): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet df16f34990c6480d93e8fe4aaec8bf17: Network error: Client connection negotiation failed: client connection to 127.27.225.193:40909: connect: Connection refused (error 111)
I20260806 21:06:33.024140 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 30671
I20260806 21:06:33.031750 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 30279
2026-08-06T21:06:33Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6772 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:06:33 dist-test-slave-69rg krb5kdc[30982](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:06:33 dist-test-slave-69rg krb5kdc[30982](info): set up 2 sockets
Aug 06 21:06:33 dist-test-slave-69rg krb5kdc[30982](info): commencing operation
krb5kdc: starting...
W20260806 21:06:34.121364 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:06:34 dist-test-slave-69rg krb5kdc[30982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:06:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:06:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:06:34.270357 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:40913
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:43603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:40913
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:34.387997 30995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:34.388321 30995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:34.388367 30995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:34.392530 30995 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:06:34.392609 30995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:34.392630 30995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:34.392648 30995 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:06:34.392663 30995 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:06:34.398195 30995 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:43603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:40913
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:40913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:34.399247 30995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:34.400184 30995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:34.409225 31000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:34.409211 31001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:34.409389 30995 server_base.cc:1034] running on GCE node
W20260806 21:06:34.409243 31003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:34.410369 30995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:34.411576 30995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:34.412781 30995 hybrid_clock.cc:648] HybridClock initialized: now 1786050394412764 us; error 36 us; skew 500 ppm
Aug 06 21:06:34 dist-test-slave-69rg krb5kdc[30982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050394, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:34.420527 30995 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:06:34.421573 30995 webserver.cc:460] Webserver started at http://127.27.225.254:45233/ using document root <none> and password file <none>
I20260806 21:06:34.422250 30995 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:34.422353 30995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:34.422608 30995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:34.424804 30995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "63bf9ae3d2044dd19f6215ced1e5abc0"
format_stamp: "Formatted at 2026-08-06 21:06:34 on dist-test-slave-69rg"
server_key: "1b588bb1961b63b34264dcbadc9b4c00"
server_key_iv: "70fc349cf89c75c44f9a8ff94f8fcb20"
server_key_version: "encryptionkey@0"
I20260806 21:06:34.425436 30995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "63bf9ae3d2044dd19f6215ced1e5abc0"
format_stamp: "Formatted at 2026-08-06 21:06:34 on dist-test-slave-69rg"
server_key: "1b588bb1961b63b34264dcbadc9b4c00"
server_key_iv: "70fc349cf89c75c44f9a8ff94f8fcb20"
server_key_version: "encryptionkey@0"
I20260806 21:06:34.429400 30995 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:06:34.432017 31010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:34.433245 30995 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:06:34.433388 30995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "63bf9ae3d2044dd19f6215ced1e5abc0"
format_stamp: "Formatted at 2026-08-06 21:06:34 on dist-test-slave-69rg"
server_key: "1b588bb1961b63b34264dcbadc9b4c00"
server_key_iv: "70fc349cf89c75c44f9a8ff94f8fcb20"
server_key_version: "encryptionkey@0"
I20260806 21:06:34.433511 30995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:34.447608 30995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:34.451212 30995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:34.451443 30995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:34.459097 30995 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:40913
I20260806 21:06:34.459102 31064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:40913 every 8 connection(s)
I20260806 21:06:34.460110 30995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:06:34.463018 31065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:34.468931 31065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63bf9ae3d2044dd19f6215ced1e5abc0: Bootstrap starting.
I20260806 21:06:34.469400 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 30995
I20260806 21:06:34.469537 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:06:34.469887 28551 external_mini_cluster.cc:1428] Setting key 3172a19bbc314999684ef690f6b1662a
I20260806 21:06:34.471853 31065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63bf9ae3d2044dd19f6215ced1e5abc0: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:34.472633 31065 log.cc:824] T 00000000000000000000000000000000 P 63bf9ae3d2044dd19f6215ced1e5abc0: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:34.475077 31065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63bf9ae3d2044dd19f6215ced1e5abc0: No bootstrap required, opened a new log
I20260806 21:06:34.479557 31065 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63bf9ae3d2044dd19f6215ced1e5abc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63bf9ae3d2044dd19f6215ced1e5abc0" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40913 } }
I20260806 21:06:34.479885 31065 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63bf9ae3d2044dd19f6215ced1e5abc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:34.479970 31065 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63bf9ae3d2044dd19f6215ced1e5abc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63bf9ae3d2044dd19f6215ced1e5abc0, State: Initialized, Role: FOLLOWER
Aug 06 21:06:34 dist-test-slave-69rg krb5kdc[30982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:34.480450 31065 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63bf9ae3d2044dd19f6215ced1e5abc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63bf9ae3d2044dd19f6215ced1e5abc0" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40913 } }
I20260806 21:06:34.480619 31065 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63bf9ae3d2044dd19f6215ced1e5abc0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:06:34.480708 31065 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63bf9ae3d2044dd19f6215ced1e5abc0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:06:34.480822 31065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63bf9ae3d2044dd19f6215ced1e5abc0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:34.481870 31065 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63bf9ae3d2044dd19f6215ced1e5abc0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63bf9ae3d2044dd19f6215ced1e5abc0" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40913 } }
I20260806 21:06:34.482292 31065 leader_election.cc:302] T 00000000000000000000000000000000 P 63bf9ae3d2044dd19f6215ced1e5abc0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63bf9ae3d2044dd19f6215ced1e5abc0; no voters:
I20260806 21:06:34.482914 31065 leader_election.cc:288] T 00000000000000000000000000000000 P 63bf9ae3d2044dd19f6215ced1e5abc0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:06:34.483091 31070 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63bf9ae3d2044dd19f6215ced1e5abc0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:34.483331 31070 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63bf9ae3d2044dd19f6215ced1e5abc0 [term 1 LEADER]: Becoming Leader. State: Replica: 63bf9ae3d2044dd19f6215ced1e5abc0, State: Running, Role: LEADER
I20260806 21:06:34.483698 31070 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63bf9ae3d2044dd19f6215ced1e5abc0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63bf9ae3d2044dd19f6215ced1e5abc0" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40913 } }
I20260806 21:06:34.484282 31065 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63bf9ae3d2044dd19f6215ced1e5abc0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:06:34.485570 31072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63bf9ae3d2044dd19f6215ced1e5abc0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63bf9ae3d2044dd19f6215ced1e5abc0. Latest consensus state: current_term: 1 leader_uuid: "63bf9ae3d2044dd19f6215ced1e5abc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63bf9ae3d2044dd19f6215ced1e5abc0" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40913 } } }
I20260806 21:06:34.485707 31072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63bf9ae3d2044dd19f6215ced1e5abc0 [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:34.485793 31071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63bf9ae3d2044dd19f6215ced1e5abc0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63bf9ae3d2044dd19f6215ced1e5abc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63bf9ae3d2044dd19f6215ced1e5abc0" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40913 } } }
I20260806 21:06:34.485975 31071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63bf9ae3d2044dd19f6215ced1e5abc0 [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:34.486306 31079 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:06:34.488291 31068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:34.471331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52510 (local address 127.27.225.254:40913)
0806 21:06:34.471832 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:34.471849 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:34.471887 (+ 38us) server_negotiation.cc:407] Connection header received
0806 21:06:34.472913 (+ 1026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:34.472937 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:34.473462 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:34.473874 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:34.474796 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:34.476036 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:34.477231 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:34.477740 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:34.477956 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:34.481673 (+ 3717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:34.481678 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:34.481692 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:34.481723 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:34.484203 (+ 2480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:34.484809 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:34.484816 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:34.484825 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:34.484991 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:34.485334 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:34.485339 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:34.485342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:34.486157 (+ 815us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:34.486354 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:34.486903 (+ 549us) server_negotiation.cc:299] Negotiation successful
0806 21:06:34.487241 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":352,"thread_start_us":173,"threads_started":1}
I20260806 21:06:34.489930 31079 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:06:34.495306 31079 catalog_manager.cc:1347] Generated new cluster ID: 0861d039d7804424be95b71459d21ba5
I20260806 21:06:34.495455 31079 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:06:34.510774 31079 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:06:34.512292 31079 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:06:34.517243 31079 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63bf9ae3d2044dd19f6215ced1e5abc0: Generated new TSK 0
I20260806 21:06:34.518105 31079 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.225.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
I20260806 21:06:34.557590 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40913
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:43603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:34.683374 31091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:34.683674 31091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:34.683777 31091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:34.688271 31091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:34.688347 31091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:34.688410 31091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:06:34.693740 31091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:43603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:34.694785 31091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:34.695744 31091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:34.705260 31099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:34.705335 31097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:34.705356 31096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:34.705768 31091 server_base.cc:1034] running on GCE node
I20260806 21:06:34.706223 31091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:34.706956 31091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:34.708194 31091 hybrid_clock.cc:648] HybridClock initialized: now 1786050394708168 us; error 41 us; skew 500 ppm
Aug 06 21:06:34 dist-test-slave-69rg krb5kdc[30982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050394, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:34.711303 31091 init.cc:377] Logged in from keytab as kudu/127.27.225.193@KRBTEST.COM (short username kudu)
I20260806 21:06:34.712277 31091 webserver.cc:460] Webserver started at http://127.27.225.193:42979/ using document root <none> and password file <none>
I20260806 21:06:34.712877 31091 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:34.712946 31091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:34.713207 31091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:34.715579 31091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "0f97fec745664912842cb95f5b1dd85a"
format_stamp: "Formatted at 2026-08-06 21:06:34 on dist-test-slave-69rg"
server_key: "d3103f2a1e2f847fc4476dd88519d786"
server_key_iv: "1b0ba8f42d4b273709adb3683d76b925"
server_key_version: "encryptionkey@0"
I20260806 21:06:34.716245 31091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "0f97fec745664912842cb95f5b1dd85a"
format_stamp: "Formatted at 2026-08-06 21:06:34 on dist-test-slave-69rg"
server_key: "d3103f2a1e2f847fc4476dd88519d786"
server_key_iv: "1b0ba8f42d4b273709adb3683d76b925"
server_key_version: "encryptionkey@0"
I20260806 21:06:34.720481 31091 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:06:34.723284 31106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:34.724678 31091 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 21:06:34.724835 31091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "0f97fec745664912842cb95f5b1dd85a"
format_stamp: "Formatted at 2026-08-06 21:06:34 on dist-test-slave-69rg"
server_key: "d3103f2a1e2f847fc4476dd88519d786"
server_key_iv: "1b0ba8f42d4b273709adb3683d76b925"
server_key_version: "encryptionkey@0"
I20260806 21:06:34.724963 31091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:34.761655 31091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:34.765089 31091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:34.765342 31091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:34.766052 31091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:34.766963 31091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:34.767011 31091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:34.767082 31091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:34.767114 31091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:34.775942 31091 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:38355
I20260806 21:06:34.776026 31221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:38355 every 8 connection(s)
I20260806 21:06:34.776906 31091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:06:34.786145 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 31091
I20260806 21:06:34.786250 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:06:34.786535 28551 external_mini_cluster.cc:1428] Setting key f93a15003405ae55ee6d47f2af33fdac
Aug 06 21:06:34 dist-test-slave-69rg krb5kdc[30982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050394, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:34.791772 31068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:34.778921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:33951 (local address 127.27.225.254:40913)
0806 21:06:34.779116 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:34.779122 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:34.779928 (+ 806us) server_negotiation.cc:407] Connection header received
0806 21:06:34.781265 (+ 1337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:34.781268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:34.781329 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:34.781436 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:34.782651 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:34.783199 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:34.784429 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:34.784667 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:34.784783 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:34.787773 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:34.787777 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:34.787779 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:34.787804 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:34.789467 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:34.790109 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:34.790113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:34.790114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:34.790197 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:34.790578 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:34.790581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:34.790583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:34.790787 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:34.790901 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:34.791511 (+ 610us) server_negotiation.cc:299] Negotiation successful
0806 21:06:34.791637 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 21:06:34.792994 31224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:34.779293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40913 (local address 127.27.225.193:33951)
0806 21:06:34.779765 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:34.779810 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:34.781041 (+ 1231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:34.781612 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:34.781619 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:34.781997 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:34.782497 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:34.782511 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:34.783333 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:34.783338 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:34.784300 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:34.784310 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:34.784823 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:34.784828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:34.785036 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:34.785689 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:34.785706 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:34.787611 (+ 1905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:34.789598 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:34.789604 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:34.789622 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:34.789992 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:34.790300 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:34.790304 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:34.790305 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:34.790471 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:34.790899 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:34.790907 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:34.791185 (+ 278us) client_negotiation.cc:769] Sending connection context
0806 21:06:34.791394 (+ 209us) client_negotiation.cc:240] Negotiation successful
0806 21:06:34.791797 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":131,"threads_started":1}
I20260806 21:06:34.794600 31222 heartbeater.cc:347] Connected to a master server at 127.27.225.254:40913
I20260806 21:06:34.794916 31222 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:34.795614 31222 heartbeater.cc:511] Master 127.27.225.254:40913 requested a full tablet report, sending...
I20260806 21:06:34.797490 31027 ts_manager.cc:194] Registered new tserver with Master: 0f97fec745664912842cb95f5b1dd85a (127.27.225.193:38355)
WARNING: no policy specified for kudu/127.27.225.194@KRBTEST.COM; defaulting to no policy
I20260806 21:06:34.798878 31027 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.193@KRBTEST.COM'} at 127.27.225.193:33951
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
I20260806 21:06:34.823175 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40913
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:43603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:34.943396 31227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:34.943671 31227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:34.943717 31227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:34.947818 31227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:34.947887 31227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:34.947978 31227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:06:34.953395 31227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:43603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:34.954509 31227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:34.955529 31227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:34.965584 31236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:34.965579 31234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:34.965555 31233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:34.965634 31227 server_base.cc:1034] running on GCE node
I20260806 21:06:34.966221 31227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:34.966802 31227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:34.967942 31227 hybrid_clock.cc:648] HybridClock initialized: now 1786050394967913 us; error 42 us; skew 500 ppm
Aug 06 21:06:34 dist-test-slave-69rg krb5kdc[30982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050394, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:34.970752 31227 init.cc:377] Logged in from keytab as kudu/127.27.225.194@KRBTEST.COM (short username kudu)
I20260806 21:06:34.971634 31227 webserver.cc:460] Webserver started at http://127.27.225.194:38399/ using document root <none> and password file <none>
I20260806 21:06:34.972183 31227 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:34.972270 31227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:34.972500 31227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:34.974620 31227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "fa5afd1a67d24571905867fc35fbb61c"
format_stamp: "Formatted at 2026-08-06 21:06:34 on dist-test-slave-69rg"
server_key: "eb494150f4c7e65b9ef5decb8ed83abc"
server_key_iv: "cd1dbc1bc2026bc8a8fac930017c0814"
server_key_version: "encryptionkey@0"
I20260806 21:06:34.975188 31227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "fa5afd1a67d24571905867fc35fbb61c"
format_stamp: "Formatted at 2026-08-06 21:06:34 on dist-test-slave-69rg"
server_key: "eb494150f4c7e65b9ef5decb8ed83abc"
server_key_iv: "cd1dbc1bc2026bc8a8fac930017c0814"
server_key_version: "encryptionkey@0"
I20260806 21:06:34.979039 31227 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260806 21:06:34.981685 31244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:34.983377 31227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 21:06:34.983553 31227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "fa5afd1a67d24571905867fc35fbb61c"
format_stamp: "Formatted at 2026-08-06 21:06:34 on dist-test-slave-69rg"
server_key: "eb494150f4c7e65b9ef5decb8ed83abc"
server_key_iv: "cd1dbc1bc2026bc8a8fac930017c0814"
server_key_version: "encryptionkey@0"
I20260806 21:06:34.983703 31227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:35.015857 31227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:35.019865 31227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:35.020107 31227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:35.020819 31227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:35.021778 31227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:35.021880 31227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:35.021965 31227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:35.022015 31227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:35.033471 31227 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:40171
I20260806 21:06:35.033602 31360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:40171 every 8 connection(s)
I20260806 21:06:35.034811 31227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:06:35.041728 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 31227
I20260806 21:06:35.041872 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:06:35.042202 28551 external_mini_cluster.cc:1428] Setting key c1636b7adeedcc71b4dff4e1a4f21096
Aug 06 21:06:35 dist-test-slave-69rg krb5kdc[30982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050394, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:35.053614 31068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:35.037064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:58423 (local address 127.27.225.254:40913)
0806 21:06:35.037226 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:35.037231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:35.038648 (+ 1417us) server_negotiation.cc:407] Connection header received
0806 21:06:35.040699 (+ 2051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:35.040703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:35.040769 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:35.040880 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:35.042464 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:35.043086 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:35.044957 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:35.045211 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:35.045339 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:35.048283 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:35.048290 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:35.048296 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:35.048336 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:35.050439 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:35.051306 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:35.051313 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:35.051317 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:35.051438 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:35.051968 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:35.051974 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:35.051978 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:35.052339 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:35.052471 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:35.053188 (+ 717us) server_negotiation.cc:299] Negotiation successful
0806 21:06:35.053404 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 21:06:35.054664 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:35.037693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40913 (local address 127.27.225.194:58423)
0806 21:06:35.038447 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:35.038513 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:35.040388 (+ 1875us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:35.041069 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:35.041076 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:35.041493 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:35.042256 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:35.042279 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:35.043445 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:35.043448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:35.044821 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:35.044832 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:35.045388 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:35.045392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:35.045579 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:35.046327 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:35.046355 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:35.048008 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:35.050741 (+ 2733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:35.050751 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:35.050775 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:35.051136 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:35.051615 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:35.051617 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:35.051619 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:35.051822 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:35.052493 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:35.052501 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:35.052863 (+ 362us) client_negotiation.cc:769] Sending connection context
0806 21:06:35.053056 (+ 193us) client_negotiation.cc:240] Negotiation successful
0806 21:06:35.053526 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":178,"threads_started":1}
WARNING: no policy specified for kudu/127.27.225.195@KRBTEST.COM; defaulting to no policy
I20260806 21:06:35.056360 31361 heartbeater.cc:347] Connected to a master server at 127.27.225.254:40913
I20260806 21:06:35.056700 31361 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:35.057328 31361 heartbeater.cc:511] Master 127.27.225.254:40913 requested a full tablet report, sending...
I20260806 21:06:35.058564 31027 ts_manager.cc:194] Registered new tserver with Master: fa5afd1a67d24571905867fc35fbb61c (127.27.225.194:40171)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.195@KRBTEST.COM" created.
I20260806 21:06:35.059474 31027 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.194@KRBTEST.COM'} at 127.27.225.194:58423
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
I20260806 21:06:35.078858 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40913
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:43603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:35.197676 31367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:35.198086 31367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:35.198174 31367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:35.202507 31367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:35.202593 31367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:35.202684 31367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:06:35.208232 31367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:43603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:35.209331 31367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:35.210433 31367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:35.220420 31373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:35.220425 31375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:35.220886 31372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:35.221189 31367 server_base.cc:1034] running on GCE node
I20260806 21:06:35.221714 31367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:35.222718 31367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:35.223906 31367 hybrid_clock.cc:648] HybridClock initialized: now 1786050395223894 us; error 35 us; skew 500 ppm
Aug 06 21:06:35 dist-test-slave-69rg krb5kdc[30982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050395, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:35.226830 31367 init.cc:377] Logged in from keytab as kudu/127.27.225.195@KRBTEST.COM (short username kudu)
I20260806 21:06:35.227788 31367 webserver.cc:460] Webserver started at http://127.27.225.195:40687/ using document root <none> and password file <none>
I20260806 21:06:35.228430 31367 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:35.228492 31367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:35.228725 31367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:35.231086 31367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "21da33b54b9a4a71b9826209063de04a"
format_stamp: "Formatted at 2026-08-06 21:06:35 on dist-test-slave-69rg"
server_key: "d64f84ece2fe9a56961ed1f0415dfa6b"
server_key_iv: "29c0b425d534bdf46b71876060ec9495"
server_key_version: "encryptionkey@0"
I20260806 21:06:35.231694 31367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "21da33b54b9a4a71b9826209063de04a"
format_stamp: "Formatted at 2026-08-06 21:06:35 on dist-test-slave-69rg"
server_key: "d64f84ece2fe9a56961ed1f0415dfa6b"
server_key_iv: "29c0b425d534bdf46b71876060ec9495"
server_key_version: "encryptionkey@0"
I20260806 21:06:35.235589 31367 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260806 21:06:35.238073 31382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:35.239372 31367 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:06:35.239498 31367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "21da33b54b9a4a71b9826209063de04a"
format_stamp: "Formatted at 2026-08-06 21:06:35 on dist-test-slave-69rg"
server_key: "d64f84ece2fe9a56961ed1f0415dfa6b"
server_key_iv: "29c0b425d534bdf46b71876060ec9495"
server_key_version: "encryptionkey@0"
I20260806 21:06:35.239610 31367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:35.259421 31367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:35.262940 31367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:35.263161 31367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:35.263803 31367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:35.264675 31367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:35.264731 31367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:35.264801 31367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:35.264842 31367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:35.274641 31367 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:33183
I20260806 21:06:35.274679 31499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:33183 every 8 connection(s)
I20260806 21:06:35.275663 31367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:06:35.276760 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 31367
I20260806 21:06:35.276893 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:06:35.277186 28551 external_mini_cluster.cc:1428] Setting key fc65aec6c8d4b07cbc34fbda6b77d041
Aug 06 21:06:35 dist-test-slave-69rg krb5kdc[30982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050395, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:35.291815 31068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:35.277942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:56387 (local address 127.27.225.254:40913)
0806 21:06:35.278121 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:35.278127 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:35.279593 (+ 1466us) server_negotiation.cc:407] Connection header received
0806 21:06:35.281123 (+ 1530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:35.281126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:35.281181 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:35.281261 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:35.282642 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:35.283194 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:35.284545 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:35.284859 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:35.284964 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:35.287915 (+ 2951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:35.287919 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:35.287921 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:35.287948 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:35.289531 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:35.290179 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:35.290182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:35.290184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:35.290264 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:35.290647 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:35.290651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:35.290652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:35.290855 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:35.290961 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:35.291516 (+ 555us) server_negotiation.cc:299] Negotiation successful
0806 21:06:35.291652 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 21:06:35.292721 31502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:35.278460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40913 (local address 127.27.225.195:56387)
0806 21:06:35.279371 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:35.279442 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:35.280839 (+ 1397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:35.281413 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:35.281421 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:35.281937 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:35.282484 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:35.282500 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:35.283321 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:35.283324 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:35.284401 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:35.284411 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:35.285027 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:35.285032 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:35.285257 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:35.286090 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:35.286110 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:35.287698 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:35.289655 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:35.289661 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:35.289678 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:35.290035 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:35.290357 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:35.290360 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:35.290362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:35.290549 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:35.290959 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:35.290966 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:35.291245 (+ 279us) client_negotiation.cc:769] Sending connection context
0806 21:06:35.291458 (+ 213us) client_negotiation.cc:240] Negotiation successful
0806 21:06:35.291821 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":197,"threads_started":1}
I20260806 21:06:35.294232 31500 heartbeater.cc:347] Connected to a master server at 127.27.225.254:40913
I20260806 21:06:35.294500 31500 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:35.295042 31500 heartbeater.cc:511] Master 127.27.225.254:40913 requested a full tablet report, sending...
I20260806 21:06:35.296181 31027 ts_manager.cc:194] Registered new tserver with Master: 21da33b54b9a4a71b9826209063de04a (127.27.225.195:33183)
I20260806 21:06:35.296779 31027 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.195@KRBTEST.COM'} at 127.27.225.195:56387
I20260806 21:06:35.304342 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 21:06:35.319778 31068 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:06:35.318176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52512 (local address 127.27.225.254:40913)
0806 21:06:35.318338 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:35.318343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:35.318519 (+ 176us) server_negotiation.cc:407] Connection header received
0806 21:06:35.318638 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:35.318641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:35.318689 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:35.318775 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:35.319654 (+ 879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52512: BlockingRecv error: recv got EOF from 127.0.0.1:52512 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 21:06:35.321079 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 31091
I20260806 21:06:35.327757 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 31227
I20260806 21:06:35.333775 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 31367
I20260806 21:06:35.340595 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 30995
2026-08-06T21:06:35Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2308 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:06:35 dist-test-slave-69rg krb5kdc[31512](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:06:35 dist-test-slave-69rg krb5kdc[31512](info): set up 2 sockets
Aug 06 21:06:35 dist-test-slave-69rg krb5kdc[31512](info): commencing operation
krb5kdc: starting...
W20260806 21:06:36.505339 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260806 21:06:36.674932 28551 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.169s user 0.000s sys 0.003s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260806 21:06:36.819953 28551 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.145s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260806 21:06:36.954295 28551 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.134s user 0.001s sys 0.001s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/krb5kdc/test-user.keytab.
W20260806 21:06:37.664320 28551 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.710s user 0.001s sys 0.001s
Aug 06 21:06:37 dist-test-slave-69rg krb5kdc[31512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:06:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:06:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
W20260806 21:06:38.058933 28551 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.27.225.254: real 0.365s user 0.000s sys 0.005s
I20260806 21:06:38.059823 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:44209
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:36977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:44209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:38.186336 31527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:38.186743 31527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:38.186823 31527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:38.192121 31527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:06:38.192299 31527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:38.192345 31527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:38.192390 31527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:06:38.192427 31527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:06:38.198796 31527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:36977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:44209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:44209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:38.200103 31527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:38.201294 31527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:38.211253 31533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:38.211199 31535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:38.211199 31532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:38.211568 31527 server_base.cc:1034] running on GCE node
I20260806 21:06:38.212363 31527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:38.213707 31527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:38.214941 31527 hybrid_clock.cc:648] HybridClock initialized: now 1786050398214893 us; error 63 us; skew 500 ppm
Aug 06 21:06:38 dist-test-slave-69rg krb5kdc[31512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050398, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:38.218513 31527 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:06:38.219683 31527 webserver.cc:460] Webserver started at http://127.27.225.254:38001/ using document root <none> and password file <none>
I20260806 21:06:38.220329 31527 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:38.220430 31527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:38.220681 31527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:38.223133 31527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "91c3a96ac42b47e6bd25c522e2322597"
format_stamp: "Formatted at 2026-08-06 21:06:38 on dist-test-slave-69rg"
server_key: "420e4f0d5a9c72a03c884d554cd38f4e"
server_key_iv: "efced02a58d23f71c4c5a88f966938f6"
server_key_version: "encryptionkey@0"
I20260806 21:06:38.223856 31527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "91c3a96ac42b47e6bd25c522e2322597"
format_stamp: "Formatted at 2026-08-06 21:06:38 on dist-test-slave-69rg"
server_key: "420e4f0d5a9c72a03c884d554cd38f4e"
server_key_iv: "efced02a58d23f71c4c5a88f966938f6"
server_key_version: "encryptionkey@0"
I20260806 21:06:38.228771 31527 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 21:06:38.232168 31543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:38.234200 31527 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260806 21:06:38.234388 31527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "91c3a96ac42b47e6bd25c522e2322597"
format_stamp: "Formatted at 2026-08-06 21:06:38 on dist-test-slave-69rg"
server_key: "420e4f0d5a9c72a03c884d554cd38f4e"
server_key_iv: "efced02a58d23f71c4c5a88f966938f6"
server_key_version: "encryptionkey@0"
I20260806 21:06:38.234575 31527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:38.251636 31527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:38.255681 31527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:38.255880 31527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:38.265105 31527 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:44209
I20260806 21:06:38.265566 31598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:44209 every 8 connection(s)
I20260806 21:06:38.266319 31527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:06:38.269811 31599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:38.270294 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 31527
I20260806 21:06:38.270418 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:06:38.270926 28551 external_mini_cluster.cc:1428] Setting key 6824652770b6588a16a2677f66f9a564
I20260806 21:06:38.279326 31599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91c3a96ac42b47e6bd25c522e2322597: Bootstrap starting.
I20260806 21:06:38.283215 31599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91c3a96ac42b47e6bd25c522e2322597: Neither blocks nor log segments found. Creating new log.
Aug 06 21:06:38 dist-test-slave-69rg krb5kdc[31512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:38.284509 31599 log.cc:824] T 00000000000000000000000000000000 P 91c3a96ac42b47e6bd25c522e2322597: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:38.287083 31599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91c3a96ac42b47e6bd25c522e2322597: No bootstrap required, opened a new log
I20260806 21:06:38.291023 31602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:38.272704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45214 (local address 127.27.225.254:44209)
0806 21:06:38.273238 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:38.273262 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:38.273317 (+ 55us) server_negotiation.cc:407] Connection header received
0806 21:06:38.274567 (+ 1250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:38.274602 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:38.275131 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:38.275588 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:38.276548 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:38.278192 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:38.279569 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:38.280315 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:38.280784 (+ 469us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:38.284641 (+ 3857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:38.284653 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:38.284677 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:38.284723 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:38.287231 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:38.288021 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:38.288034 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:38.288040 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:38.288316 (+ 276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:38.288702 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:38.288706 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:38.288708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:38.289196 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:38.289419 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:38.289791 (+ 372us) server_negotiation.cc:299] Negotiation successful
0806 21:06:38.290181 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":136,"threads_started":1}
I20260806 21:06:38.291728 31599 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91c3a96ac42b47e6bd25c522e2322597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c3a96ac42b47e6bd25c522e2322597" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44209 } }
I20260806 21:06:38.292001 31599 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91c3a96ac42b47e6bd25c522e2322597 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:38.292862 31599 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91c3a96ac42b47e6bd25c522e2322597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c3a96ac42b47e6bd25c522e2322597, State: Initialized, Role: FOLLOWER
I20260806 21:06:38.293455 31599 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91c3a96ac42b47e6bd25c522e2322597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c3a96ac42b47e6bd25c522e2322597" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44209 } }
I20260806 21:06:38.293651 31599 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91c3a96ac42b47e6bd25c522e2322597 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:06:38.293725 31599 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91c3a96ac42b47e6bd25c522e2322597 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:06:38.294214 31599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91c3a96ac42b47e6bd25c522e2322597 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:38.294785 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:06:38.296029 31599 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91c3a96ac42b47e6bd25c522e2322597 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c3a96ac42b47e6bd25c522e2322597" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44209 } }
I20260806 21:06:38.296550 31599 leader_election.cc:302] T 00000000000000000000000000000000 P 91c3a96ac42b47e6bd25c522e2322597 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91c3a96ac42b47e6bd25c522e2322597; no voters:
I20260806 21:06:38.297017 31599 leader_election.cc:288] T 00000000000000000000000000000000 P 91c3a96ac42b47e6bd25c522e2322597 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:06:38.297130 31604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91c3a96ac42b47e6bd25c522e2322597 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:38.297363 31604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91c3a96ac42b47e6bd25c522e2322597 [term 1 LEADER]: Becoming Leader. State: Replica: 91c3a96ac42b47e6bd25c522e2322597, State: Running, Role: LEADER
I20260806 21:06:38.297775 31604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91c3a96ac42b47e6bd25c522e2322597 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c3a96ac42b47e6bd25c522e2322597" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44209 } }
I20260806 21:06:38.298660 31599 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91c3a96ac42b47e6bd25c522e2322597 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:06:38.299249 31604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91c3a96ac42b47e6bd25c522e2322597 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91c3a96ac42b47e6bd25c522e2322597. Latest consensus state: current_term: 1 leader_uuid: "91c3a96ac42b47e6bd25c522e2322597" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c3a96ac42b47e6bd25c522e2322597" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44209 } } }
I20260806 21:06:38.299391 31604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91c3a96ac42b47e6bd25c522e2322597 [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:38.299875 31605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91c3a96ac42b47e6bd25c522e2322597 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91c3a96ac42b47e6bd25c522e2322597" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c3a96ac42b47e6bd25c522e2322597" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44209 } } }
I20260806 21:06:38.299988 31605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91c3a96ac42b47e6bd25c522e2322597 [sys.catalog]: This master's current role is: LEADER
W20260806 21:06:38.304883 31621 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 91c3a96ac42b47e6bd25c522e2322597: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:06:38.304999 31621 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:06:38.305078 31609 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:06:38.306103 31609 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:06:38.314824 31609 catalog_manager.cc:1347] Generated new cluster ID: 7c11bf0276f041a1b6520a42bb993059
I20260806 21:06:38.314916 31609 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:06:38.320744 31602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:38.313170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45222 (local address 127.27.225.254:44209)
0806 21:06:38.313301 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:38.313306 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:38.313760 (+ 454us) server_negotiation.cc:407] Connection header received
0806 21:06:38.314022 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:38.314025 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:38.314084 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:38.314178 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:38.315043 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:38.315933 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:38.317383 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:38.317946 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:38.318115 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:38.319266 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:38.319271 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:38.319273 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:06:38.320078 (+ 805us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:38.320133 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:06:38.320159 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:38.320329 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:38.320473 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 21:06:38.320560 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 21:06:38.325634 31609 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:06:38.327777 31609 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:06:38.336323 31609 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91c3a96ac42b47e6bd25c522e2322597: Generated new TSK 0
I20260806 21:06:38.337365 31609 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:06:38.346048 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 31527
2026-08-06T21:06:38Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (3004 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:06:38 dist-test-slave-69rg krb5kdc[31633](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:06:38 dist-test-slave-69rg krb5kdc[31633](info): set up 2 sockets
Aug 06 21:06:38 dist-test-slave-69rg krb5kdc[31633](info): commencing operation
krb5kdc: starting...
W20260806 21:06:39.462170 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:06:39 dist-test-slave-69rg krb5kdc[31633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:06:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:06:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:06:39.601119 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:37463
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:40605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:37463
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:39.715682 31648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:39.715971 31648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:39.716017 31648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:39.720522 31648 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:06:39.720616 31648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:39.720638 31648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:39.720656 31648 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:06:39.720674 31648 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:06:39.726465 31648 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:40605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:37463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:37463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:39.727639 31648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:39.728695 31648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:39.737447 31655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:39.737489 31653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:39.737489 31657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:39.737762 31648 server_base.cc:1034] running on GCE node
I20260806 21:06:39.738411 31648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:39.739607 31648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:39.740790 31648 hybrid_clock.cc:648] HybridClock initialized: now 1786050399740759 us; error 45 us; skew 500 ppm
Aug 06 21:06:39 dist-test-slave-69rg krb5kdc[31633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050399, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:39.747385 31648 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:06:39.748456 31648 webserver.cc:460] Webserver started at http://127.27.225.254:33171/ using document root <none> and password file <none>
I20260806 21:06:39.749109 31648 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:39.749208 31648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:39.749467 31648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:39.751820 31648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "95a31ec48a78442c92f3007bdd3b520f"
format_stamp: "Formatted at 2026-08-06 21:06:39 on dist-test-slave-69rg"
server_key: "400d37b1923e9a8a520847534a980bef"
server_key_iv: "9d9b58454f34bc861cbe8ee8608e4d69"
server_key_version: "encryptionkey@0"
I20260806 21:06:39.752468 31648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "95a31ec48a78442c92f3007bdd3b520f"
format_stamp: "Formatted at 2026-08-06 21:06:39 on dist-test-slave-69rg"
server_key: "400d37b1923e9a8a520847534a980bef"
server_key_iv: "9d9b58454f34bc861cbe8ee8608e4d69"
server_key_version: "encryptionkey@0"
I20260806 21:06:39.756924 31648 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 21:06:39.759729 31664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:39.760994 31648 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:06:39.761162 31648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "95a31ec48a78442c92f3007bdd3b520f"
format_stamp: "Formatted at 2026-08-06 21:06:39 on dist-test-slave-69rg"
server_key: "400d37b1923e9a8a520847534a980bef"
server_key_iv: "9d9b58454f34bc861cbe8ee8608e4d69"
server_key_version: "encryptionkey@0"
I20260806 21:06:39.761304 31648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:39.784513 31648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:39.787930 31648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:39.788160 31648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:39.796381 31648 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:37463
I20260806 21:06:39.796705 31719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:37463 every 8 connection(s)
I20260806 21:06:39.797463 31648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:06:39.798784 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 31648
I20260806 21:06:39.798969 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:06:39.799276 28551 external_mini_cluster.cc:1428] Setting key 6a271d9bb814b0a078226d7960b221c5
I20260806 21:06:39.800839 31720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:39.807955 31720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95a31ec48a78442c92f3007bdd3b520f: Bootstrap starting.
Aug 06 21:06:39 dist-test-slave-69rg krb5kdc[31633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:39.810822 31720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95a31ec48a78442c92f3007bdd3b520f: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:39.811815 31720 log.cc:824] T 00000000000000000000000000000000 P 95a31ec48a78442c92f3007bdd3b520f: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:39.814145 31720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95a31ec48a78442c92f3007bdd3b520f: No bootstrap required, opened a new log
I20260806 21:06:39.815500 31723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:39.800920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36536 (local address 127.27.225.254:37463)
0806 21:06:39.801375 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:39.801394 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:39.801438 (+ 44us) server_negotiation.cc:407] Connection header received
0806 21:06:39.802317 (+ 879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:39.802342 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:39.802758 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:39.803117 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:39.803859 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:39.805358 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:39.806346 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:39.807072 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:39.807235 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:39.810091 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:39.810098 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:39.810115 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:39.810151 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:39.812397 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:39.812909 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:39.812920 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:39.812925 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:39.813043 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:39.813356 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:39.813361 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:39.813363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:39.813795 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:39.814073 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:39.814397 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 21:06:39.814758 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":122,"threads_started":1}
I20260806 21:06:39.818300 31720 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95a31ec48a78442c92f3007bdd3b520f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95a31ec48a78442c92f3007bdd3b520f" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37463 } }
I20260806 21:06:39.818558 31720 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95a31ec48a78442c92f3007bdd3b520f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:39.818640 31720 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95a31ec48a78442c92f3007bdd3b520f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95a31ec48a78442c92f3007bdd3b520f, State: Initialized, Role: FOLLOWER
I20260806 21:06:39.819065 31720 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95a31ec48a78442c92f3007bdd3b520f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95a31ec48a78442c92f3007bdd3b520f" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37463 } }
I20260806 21:06:39.819233 31720 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95a31ec48a78442c92f3007bdd3b520f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:06:39.819324 31720 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95a31ec48a78442c92f3007bdd3b520f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:06:39.819445 31720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95a31ec48a78442c92f3007bdd3b520f [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:39.820441 31720 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95a31ec48a78442c92f3007bdd3b520f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95a31ec48a78442c92f3007bdd3b520f" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37463 } }
I20260806 21:06:39.820792 31720 leader_election.cc:302] T 00000000000000000000000000000000 P 95a31ec48a78442c92f3007bdd3b520f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95a31ec48a78442c92f3007bdd3b520f; no voters:
I20260806 21:06:39.821149 31720 leader_election.cc:288] T 00000000000000000000000000000000 P 95a31ec48a78442c92f3007bdd3b520f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:06:39.821354 31725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95a31ec48a78442c92f3007bdd3b520f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:39.821681 31725 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95a31ec48a78442c92f3007bdd3b520f [term 1 LEADER]: Becoming Leader. State: Replica: 95a31ec48a78442c92f3007bdd3b520f, State: Running, Role: LEADER
I20260806 21:06:39.822064 31725 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95a31ec48a78442c92f3007bdd3b520f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95a31ec48a78442c92f3007bdd3b520f" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37463 } }
I20260806 21:06:39.822531 31720 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95a31ec48a78442c92f3007bdd3b520f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:06:39.823985 31727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95a31ec48a78442c92f3007bdd3b520f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95a31ec48a78442c92f3007bdd3b520f. Latest consensus state: current_term: 1 leader_uuid: "95a31ec48a78442c92f3007bdd3b520f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95a31ec48a78442c92f3007bdd3b520f" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37463 } } }
I20260806 21:06:39.823995 31726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95a31ec48a78442c92f3007bdd3b520f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95a31ec48a78442c92f3007bdd3b520f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95a31ec48a78442c92f3007bdd3b520f" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37463 } } }
I20260806 21:06:39.824141 31727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95a31ec48a78442c92f3007bdd3b520f [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:39.824139 31726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95a31ec48a78442c92f3007bdd3b520f [sys.catalog]: This master's current role is: LEADER
W20260806 21:06:39.827579 31740 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 95a31ec48a78442c92f3007bdd3b520f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:06:39.827677 31740 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:06:39.827785 31735 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:06:39.828477 31735 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:06:39.833518 31735 catalog_manager.cc:1347] Generated new cluster ID: 003f2785869747f7865fddb188730c8e
I20260806 21:06:39.833592 31735 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:06:39.851630 31735 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:06:39.852974 31735 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:06:39.858443 31735 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95a31ec48a78442c92f3007bdd3b520f: Generated new TSK 0
I20260806 21:06:39.859159 31735 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.225.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
I20260806 21:06:39.893553 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:37463
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:40605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:40.028762 31747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:40.029143 31747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:40.029230 31747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:40.034247 31747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:40.034381 31747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:40.034487 31747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:06:40.043237 31747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:40605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:37463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:40.044638 31747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:40.046793 31747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:40.058509 31753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:40.058607 31755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:40.058521 31752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:40.059118 31747 server_base.cc:1034] running on GCE node
I20260806 21:06:40.059603 31747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:40.060340 31747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:40.061542 31747 hybrid_clock.cc:648] HybridClock initialized: now 1786050400061519 us; error 44 us; skew 500 ppm
Aug 06 21:06:40 dist-test-slave-69rg krb5kdc[31633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050400, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:40.065785 31747 init.cc:377] Logged in from keytab as kudu/127.27.225.193@KRBTEST.COM (short username kudu)
I20260806 21:06:40.067070 31747 webserver.cc:460] Webserver started at http://127.27.225.193:45929/ using document root <none> and password file <none>
I20260806 21:06:40.068004 31747 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:40.068118 31747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:40.068403 31747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:40.071452 31747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "95902823186a444085cd162f732f5771"
format_stamp: "Formatted at 2026-08-06 21:06:40 on dist-test-slave-69rg"
server_key: "704d116d6909d74d250be9a279139b2e"
server_key_iv: "50e10c586c171ff1436014f6ee558cdd"
server_key_version: "encryptionkey@0"
I20260806 21:06:40.072139 31747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "95902823186a444085cd162f732f5771"
format_stamp: "Formatted at 2026-08-06 21:06:40 on dist-test-slave-69rg"
server_key: "704d116d6909d74d250be9a279139b2e"
server_key_iv: "50e10c586c171ff1436014f6ee558cdd"
server_key_version: "encryptionkey@0"
I20260806 21:06:40.076504 31747 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:06:40.079566 31762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:40.081225 31747 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 21:06:40.081418 31747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "95902823186a444085cd162f732f5771"
format_stamp: "Formatted at 2026-08-06 21:06:40 on dist-test-slave-69rg"
server_key: "704d116d6909d74d250be9a279139b2e"
server_key_iv: "50e10c586c171ff1436014f6ee558cdd"
server_key_version: "encryptionkey@0"
I20260806 21:06:40.081580 31747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:40.098091 31747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:40.101783 31747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:40.102092 31747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:40.102756 31747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:40.103673 31747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:40.103750 31747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:40.103825 31747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:40.103878 31747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:40.113710 31747 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:38673
I20260806 21:06:40.113785 31884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:38673 every 8 connection(s)
I20260806 21:06:40.114804 31747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:06:40.122547 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 31747
I20260806 21:06:40.122694 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:06:40.123014 28551 external_mini_cluster.cc:1428] Setting key 5a673b474323fd670f21c3885339b104
Aug 06 21:06:40 dist-test-slave-69rg krb5kdc[31633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050400, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:40.132647 31723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:40.116987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:35127 (local address 127.27.225.254:37463)
0806 21:06:40.117196 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:40.117200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:40.118224 (+ 1024us) server_negotiation.cc:407] Connection header received
0806 21:06:40.119627 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:40.119631 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:40.119696 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:40.119782 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:40.121055 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:40.121636 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:40.123432 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:40.124287 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:40.124408 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:40.127780 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:40.127787 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:40.127791 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:40.127833 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:40.129807 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:40.130527 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:40.130533 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:40.130537 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:40.130654 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:40.131075 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:40.131079 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:40.131081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:40.131411 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:40.131543 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:40.132145 (+ 602us) server_negotiation.cc:299] Negotiation successful
0806 21:06:40.132380 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260806 21:06:40.133392 31887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:40.117398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:37463 (local address 127.27.225.193:35127)
0806 21:06:40.118040 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:40.118100 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:40.119389 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:40.119982 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:40.119989 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:40.120385 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:40.120886 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:40.120900 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:40.121781 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:40.121785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:40.123242 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:40.123263 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:40.124514 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:40.124522 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:40.124834 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:40.125631 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:40.125650 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:40.127526 (+ 1876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:40.130017 (+ 2491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:40.130023 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:40.130040 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:40.130375 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:40.130770 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:40.130772 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:40.130774 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:40.130947 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:40.131547 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:40.131554 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:40.131852 (+ 298us) client_negotiation.cc:769] Sending connection context
0806 21:06:40.132030 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 21:06:40.132385 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":127,"threads_started":1}
WARNING: no policy specified for kudu/127.27.225.194@KRBTEST.COM; defaulting to no policy
I20260806 21:06:40.135206 31885 heartbeater.cc:347] Connected to a master server at 127.27.225.254:37463
I20260806 21:06:40.135515 31885 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:40.136098 31885 heartbeater.cc:511] Master 127.27.225.254:37463 requested a full tablet report, sending...
I20260806 21:06:40.138322 31682 ts_manager.cc:194] Registered new tserver with Master: 95902823186a444085cd162f732f5771 (127.27.225.193:38673)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.194@KRBTEST.COM" created.
I20260806 21:06:40.140041 31682 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.193@KRBTEST.COM'} at 127.27.225.193:35127
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
I20260806 21:06:40.161485 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:37463
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:40605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:40.288944 31890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:40.289287 31890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:40.289335 31890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:40.293948 31890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:40.294040 31890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:40.294104 31890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:06:40.301463 31890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:40605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:37463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:40.302837 31890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:40.304021 31890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:40.315442 31895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:40.315442 31896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:40.315472 31898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:40.315878 31890 server_base.cc:1034] running on GCE node
I20260806 21:06:40.316457 31890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:40.317210 31890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:40.318423 31890 hybrid_clock.cc:648] HybridClock initialized: now 1786050400318388 us; error 52 us; skew 500 ppm
Aug 06 21:06:40 dist-test-slave-69rg krb5kdc[31633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050400, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:40.322142 31890 init.cc:377] Logged in from keytab as kudu/127.27.225.194@KRBTEST.COM (short username kudu)
I20260806 21:06:40.323827 31890 webserver.cc:460] Webserver started at http://127.27.225.194:41551/ using document root <none> and password file <none>
I20260806 21:06:40.324635 31890 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:40.324728 31890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:40.325265 31890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:40.328038 31890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "aadfe37b60fa48399e132f6f59774922"
format_stamp: "Formatted at 2026-08-06 21:06:40 on dist-test-slave-69rg"
server_key: "035e902030aeb020af7bde7244f8fde0"
server_key_iv: "6c2a0c1a653c9daa275a72af4b73a199"
server_key_version: "encryptionkey@0"
I20260806 21:06:40.328712 31890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "aadfe37b60fa48399e132f6f59774922"
format_stamp: "Formatted at 2026-08-06 21:06:40 on dist-test-slave-69rg"
server_key: "035e902030aeb020af7bde7244f8fde0"
server_key_iv: "6c2a0c1a653c9daa275a72af4b73a199"
server_key_version: "encryptionkey@0"
I20260806 21:06:40.333313 31890 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 21:06:40.336542 31905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:40.338517 31890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 21:06:40.338681 31890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "aadfe37b60fa48399e132f6f59774922"
format_stamp: "Formatted at 2026-08-06 21:06:40 on dist-test-slave-69rg"
server_key: "035e902030aeb020af7bde7244f8fde0"
server_key_iv: "6c2a0c1a653c9daa275a72af4b73a199"
server_key_version: "encryptionkey@0"
I20260806 21:06:40.338809 31890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:40.359932 31890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:40.363760 31890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:40.364012 31890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:40.364717 31890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:40.365720 31890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:40.365787 31890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:40.365869 31890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:40.365911 31890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:40.375641 31890 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:45197
I20260806 21:06:40.375813 32020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:45197 every 8 connection(s)
I20260806 21:06:40.376739 31890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:06:40.381579 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 31890
I20260806 21:06:40.381742 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:06:40.382293 28551 external_mini_cluster.cc:1428] Setting key 2974ba0a1a849a0a8551f4586ed2d7ca
Aug 06 21:06:40 dist-test-slave-69rg krb5kdc[31633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050400, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.225.195@KRBTEST.COM; defaulting to no policy
I20260806 21:06:40.396014 31723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:40.378942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:44179 (local address 127.27.225.254:37463)
0806 21:06:40.379094 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:40.379098 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:40.380136 (+ 1038us) server_negotiation.cc:407] Connection header received
0806 21:06:40.382340 (+ 2204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:40.382343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:40.382466 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:40.382553 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:40.384310 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:40.384966 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:40.387207 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:40.387723 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:40.387877 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:40.390821 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:40.390829 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:40.390836 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:40.390883 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:40.392924 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:40.393701 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:40.393706 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:40.393708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:40.393827 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:40.394453 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:40.394459 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:40.394463 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:40.394836 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:40.394956 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:40.395659 (+ 703us) server_negotiation.cc:299] Negotiation successful
0806 21:06:40.395827 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 21:06:40.396894 32023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:40.379390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:37463 (local address 127.27.225.194:44179)
0806 21:06:40.379932 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:40.379985 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:40.381802 (+ 1817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:40.382922 (+ 1120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:40.382934 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:40.383425 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:40.384141 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:40.384159 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:40.385134 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:40.385140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:40.387034 (+ 1894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:40.387049 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:40.387937 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:40.387940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:40.388114 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:40.388839 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:40.388860 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:40.390564 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:40.393090 (+ 2526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:40.393097 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:40.393153 (+ 56us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:40.393525 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:40.394003 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:40.394009 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:40.394013 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:40.394247 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:40.395003 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:40.395010 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:40.395308 (+ 298us) client_negotiation.cc:769] Sending connection context
0806 21:06:40.395588 (+ 280us) client_negotiation.cc:240] Negotiation successful
0806 21:06:40.395969 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":123,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.195@KRBTEST.COM" created.
I20260806 21:06:40.398227 32021 heartbeater.cc:347] Connected to a master server at 127.27.225.254:37463
I20260806 21:06:40.398532 32021 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:40.399127 32021 heartbeater.cc:511] Master 127.27.225.254:37463 requested a full tablet report, sending...
I20260806 21:06:40.400729 31682 ts_manager.cc:194] Registered new tserver with Master: aadfe37b60fa48399e132f6f59774922 (127.27.225.194:45197)
I20260806 21:06:40.401608 31682 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.194@KRBTEST.COM'} at 127.27.225.194:44179
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
I20260806 21:06:40.419224 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:37463
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:40605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:40.548406 32026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:40.548733 32026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:40.548779 32026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:40.553193 32026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:40.553292 32026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:40.553354 32026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:06:40.559470 32026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:40605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:37463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:40.560787 32026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:40.562292 32026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:40.573975 32032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:40.574180 32026 server_base.cc:1034] running on GCE node
W20260806 21:06:40.573939 32031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:40.574012 32034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:40.574770 32026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:40.575456 32026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:40.576738 32026 hybrid_clock.cc:648] HybridClock initialized: now 1786050400576699 us; error 48 us; skew 500 ppm
Aug 06 21:06:40 dist-test-slave-69rg krb5kdc[31633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050400, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:40.580425 32026 init.cc:377] Logged in from keytab as kudu/127.27.225.195@KRBTEST.COM (short username kudu)
I20260806 21:06:40.581653 32026 webserver.cc:460] Webserver started at http://127.27.225.195:39337/ using document root <none> and password file <none>
I20260806 21:06:40.582456 32026 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:40.582597 32026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:40.582877 32026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:40.585165 32026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "8eb49d4276ba45c0a00ad5a76f0a02ce"
format_stamp: "Formatted at 2026-08-06 21:06:40 on dist-test-slave-69rg"
server_key: "583a22e8dd474ca39bfb101aad497eb2"
server_key_iv: "5943410cbed1e426efc56fe326015e62"
server_key_version: "encryptionkey@0"
I20260806 21:06:40.585990 32026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "8eb49d4276ba45c0a00ad5a76f0a02ce"
format_stamp: "Formatted at 2026-08-06 21:06:40 on dist-test-slave-69rg"
server_key: "583a22e8dd474ca39bfb101aad497eb2"
server_key_iv: "5943410cbed1e426efc56fe326015e62"
server_key_version: "encryptionkey@0"
I20260806 21:06:40.590754 32026 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 21:06:40.594179 32041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:40.596292 32026 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:06:40.596545 32026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "8eb49d4276ba45c0a00ad5a76f0a02ce"
format_stamp: "Formatted at 2026-08-06 21:06:40 on dist-test-slave-69rg"
server_key: "583a22e8dd474ca39bfb101aad497eb2"
server_key_iv: "5943410cbed1e426efc56fe326015e62"
server_key_version: "encryptionkey@0"
I20260806 21:06:40.596726 32026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:40.625077 32026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:40.630574 32026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:40.630826 32026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:40.631539 32026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:40.632540 32026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:40.632627 32026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:40.632702 32026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:40.632752 32026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:40.646860 32026 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:41539
I20260806 21:06:40.646880 32161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:41539 every 8 connection(s)
I20260806 21:06:40.648069 32026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:06:40.649187 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 32026
I20260806 21:06:40.649338 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:06:40.649665 28551 external_mini_cluster.cc:1428] Setting key 721008c2f76d6689b1d13a3087635498
Aug 06 21:06:40 dist-test-slave-69rg krb5kdc[31633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050400, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:40.668555 31723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:40.651885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:49475 (local address 127.27.225.254:37463)
0806 21:06:40.652021 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:40.652026 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:40.652042 (+ 16us) server_negotiation.cc:407] Connection header received
0806 21:06:40.653642 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:40.653645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:40.653698 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:40.654151 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:40.655803 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:40.656350 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:40.657671 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:40.658261 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:40.658373 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:40.662364 (+ 3991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:40.662367 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:40.662370 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:40.662404 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:40.664876 (+ 2472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:40.666065 (+ 1189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:40.666073 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:40.666078 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:40.666399 (+ 321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:40.666929 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:40.666937 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:40.666945 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:40.667180 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:40.667272 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:40.668251 (+ 979us) server_negotiation.cc:299] Negotiation successful
0806 21:06:40.668423 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 21:06:40.670006 32165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:40.650909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:37463 (local address 127.27.225.195:49475)
0806 21:06:40.651598 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:40.651692 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:40.653325 (+ 1633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:40.654410 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:40.654419 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:40.654944 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:40.655568 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:40.655585 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:40.656518 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:40.656521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:40.657494 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:40.657505 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:40.658528 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:40.658532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:40.658871 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:40.659612 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:40.659630 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:40.661998 (+ 2368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:40.665245 (+ 3247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:40.665258 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:40.665281 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:40.665783 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:40.666555 (+ 772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:40.666557 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:40.666559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:40.666763 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:40.667474 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:40.667553 (+ 79us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:40.667928 (+ 375us) client_negotiation.cc:769] Sending connection context
0806 21:06:40.668132 (+ 204us) client_negotiation.cc:240] Negotiation successful
0806 21:06:40.668476 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":135,"threads_started":1}
I20260806 21:06:40.671775 32162 heartbeater.cc:347] Connected to a master server at 127.27.225.254:37463
I20260806 21:06:40.672119 32162 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:40.672771 32162 heartbeater.cc:511] Master 127.27.225.254:37463 requested a full tablet report, sending...
I20260806 21:06:40.674150 31682 ts_manager.cc:194] Registered new tserver with Master: 8eb49d4276ba45c0a00ad5a76f0a02ce (127.27.225.195:41539)
I20260806 21:06:40.675310 31682 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.195@KRBTEST.COM'} at 127.27.225.195:49475
I20260806 21:06:40.678915 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 21:06:40 dist-test-slave-69rg krb5kdc[31633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050400, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 21:06:40 dist-test-slave-69rg krb5kdc[31633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050400, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:40.716024 31723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:40.704507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36544 (local address 127.27.225.254:37463)
0806 21:06:40.704811 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:40.704817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:40.704963 (+ 146us) server_negotiation.cc:407] Connection header received
0806 21:06:40.705161 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:40.705164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:40.705219 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:40.705314 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:40.706202 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:40.706875 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:40.707955 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:40.708197 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:40.708348 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:40.711742 (+ 3394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:40.711745 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:40.711748 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:40.711786 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:40.713601 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:40.714261 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:40.714265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:40.714266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:40.714363 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:40.714718 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:40.714722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:40.714723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:40.714993 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:40.715163 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:40.715425 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 21:06:40.715568 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159}
W20260806 21:06:40.716830 31682 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36544
I20260806 21:06:40.718449 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 31747
I20260806 21:06:40.727527 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 31890
I20260806 21:06:40.735174 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 32026
I20260806 21:06:40.743800 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 31648
2026-08-06T21:06:40Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2399 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:06:40 dist-test-slave-69rg krb5kdc[32175](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:06:40 dist-test-slave-69rg krb5kdc[32175](info): set up 2 sockets
Aug 06 21:06:40 dist-test-slave-69rg krb5kdc[32175](info): commencing operation
krb5kdc: starting...
W20260806 21:06:41.842269 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:06:41 dist-test-slave-69rg krb5kdc[32175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:06:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:06:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:06:41.985385 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:41973
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:41973
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:42.109943 32189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:42.110301 32189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:42.110378 32189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:42.114571 32189 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:06:42.114691 32189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:42.114729 32189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:42.114768 32189 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:06:42.114806 32189 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:06:42.120581 32189 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:41973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:41973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:42.121954 32189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:42.123056 32189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:42.132211 32195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:42.132267 32198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:42.132211 32196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:42.133309 32189 server_base.cc:1034] running on GCE node
I20260806 21:06:42.134335 32189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:42.135632 32189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:42.136833 32189 hybrid_clock.cc:648] HybridClock initialized: now 1786050402136803 us; error 48 us; skew 500 ppm
Aug 06 21:06:42 dist-test-slave-69rg krb5kdc[32175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050402, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:42.140147 32189 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:06:42.141330 32189 webserver.cc:460] Webserver started at http://127.27.225.254:44915/ using document root <none> and password file <none>
I20260806 21:06:42.142071 32189 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:42.142184 32189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:42.142462 32189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:42.144834 32189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "3c51338fe0b7413381d5c684f8868d84"
format_stamp: "Formatted at 2026-08-06 21:06:42 on dist-test-slave-69rg"
server_key: "bfa6049a42729c4374d659cf31d0c9d2"
server_key_iv: "134efb928903643a687c5b4ad68ec5e1"
server_key_version: "encryptionkey@0"
I20260806 21:06:42.145511 32189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "3c51338fe0b7413381d5c684f8868d84"
format_stamp: "Formatted at 2026-08-06 21:06:42 on dist-test-slave-69rg"
server_key: "bfa6049a42729c4374d659cf31d0c9d2"
server_key_iv: "134efb928903643a687c5b4ad68ec5e1"
server_key_version: "encryptionkey@0"
I20260806 21:06:42.149950 32189 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.003s
I20260806 21:06:42.152702 32205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:42.154287 32189 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:06:42.154487 32189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "3c51338fe0b7413381d5c684f8868d84"
format_stamp: "Formatted at 2026-08-06 21:06:42 on dist-test-slave-69rg"
server_key: "bfa6049a42729c4374d659cf31d0c9d2"
server_key_iv: "134efb928903643a687c5b4ad68ec5e1"
server_key_version: "encryptionkey@0"
I20260806 21:06:42.154647 32189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:42.190276 32189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:42.194228 32189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:42.194485 32189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:42.202816 32189 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:41973
I20260806 21:06:42.202857 32261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:41973 every 8 connection(s)
I20260806 21:06:42.203929 32189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:06:42.204166 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 32189
I20260806 21:06:42.204295 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:06:42.204613 28551 external_mini_cluster.cc:1428] Setting key 958c2eb06858b6695efc73e51bfae3f8
I20260806 21:06:42.207696 32262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 21:06:42 dist-test-slave-69rg krb5kdc[32175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:42.215147 32262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c51338fe0b7413381d5c684f8868d84: Bootstrap starting.
I20260806 21:06:42.217633 32262 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c51338fe0b7413381d5c684f8868d84: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:42.218858 32262 log.cc:824] T 00000000000000000000000000000000 P 3c51338fe0b7413381d5c684f8868d84: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:42.221253 32262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c51338fe0b7413381d5c684f8868d84: No bootstrap required, opened a new log
I20260806 21:06:42.221266 32265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:42.206128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53238 (local address 127.27.225.254:41973)
0806 21:06:42.206700 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:42.206723 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:42.206772 (+ 49us) server_negotiation.cc:407] Connection header received
0806 21:06:42.207731 (+ 959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:42.207756 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:42.208196 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:42.208603 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:42.209340 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:42.211083 (+ 1743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:42.212020 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:42.212803 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:42.212986 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:42.215780 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:42.215785 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:42.215799 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:42.215829 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:42.217612 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:42.218279 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:42.218285 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:42.218290 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:42.218474 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:42.218861 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:42.218864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:42.218866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:42.219454 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:42.219697 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:42.220056 (+ 359us) server_negotiation.cc:299] Negotiation successful
0806 21:06:42.220490 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":161,"threads_started":1}
I20260806 21:06:42.225436 32262 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c51338fe0b7413381d5c684f8868d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c51338fe0b7413381d5c684f8868d84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41973 } }
I20260806 21:06:42.225696 32262 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c51338fe0b7413381d5c684f8868d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:42.225785 32262 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c51338fe0b7413381d5c684f8868d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c51338fe0b7413381d5c684f8868d84, State: Initialized, Role: FOLLOWER
I20260806 21:06:42.226297 32262 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c51338fe0b7413381d5c684f8868d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c51338fe0b7413381d5c684f8868d84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41973 } }
I20260806 21:06:42.226431 32262 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c51338fe0b7413381d5c684f8868d84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:06:42.226523 32262 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c51338fe0b7413381d5c684f8868d84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:06:42.226629 32262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c51338fe0b7413381d5c684f8868d84 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:42.227669 32262 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c51338fe0b7413381d5c684f8868d84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c51338fe0b7413381d5c684f8868d84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41973 } }
I20260806 21:06:42.228053 32262 leader_election.cc:302] T 00000000000000000000000000000000 P 3c51338fe0b7413381d5c684f8868d84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c51338fe0b7413381d5c684f8868d84; no voters:
I20260806 21:06:42.228403 32262 leader_election.cc:288] T 00000000000000000000000000000000 P 3c51338fe0b7413381d5c684f8868d84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:06:42.228574 32267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c51338fe0b7413381d5c684f8868d84 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:42.228873 32267 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c51338fe0b7413381d5c684f8868d84 [term 1 LEADER]: Becoming Leader. State: Replica: 3c51338fe0b7413381d5c684f8868d84, State: Running, Role: LEADER
I20260806 21:06:42.229187 32267 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c51338fe0b7413381d5c684f8868d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c51338fe0b7413381d5c684f8868d84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41973 } }
I20260806 21:06:42.229935 32262 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c51338fe0b7413381d5c684f8868d84 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:06:42.231170 32269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c51338fe0b7413381d5c684f8868d84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c51338fe0b7413381d5c684f8868d84. Latest consensus state: current_term: 1 leader_uuid: "3c51338fe0b7413381d5c684f8868d84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c51338fe0b7413381d5c684f8868d84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41973 } } }
I20260806 21:06:42.231316 32269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c51338fe0b7413381d5c684f8868d84 [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:42.231170 32268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c51338fe0b7413381d5c684f8868d84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c51338fe0b7413381d5c684f8868d84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c51338fe0b7413381d5c684f8868d84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41973 } } }
I20260806 21:06:42.231675 32268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c51338fe0b7413381d5c684f8868d84 [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:42.231834 32276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:06:42.235183 32276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:06:42.241245 32276 catalog_manager.cc:1347] Generated new cluster ID: 3cfa1b6f5d254dcba5c85741db63b199
I20260806 21:06:42.241356 32276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:06:42.258272 32276 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:06:42.259482 32276 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:06:42.278800 32276 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c51338fe0b7413381d5c684f8868d84: Generated new TSK 0
I20260806 21:06:42.279696 32276 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.225.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
I20260806 21:06:42.318161 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:41973
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:42.459066 32289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:42.459380 32289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:42.459460 32289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:42.464124 32289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:42.464205 32289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:42.464272 32289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:06:42.470839 32289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:41973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:42.472129 32289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:42.473292 32289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:42.484629 32297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:42.484690 32289 server_base.cc:1034] running on GCE node
W20260806 21:06:42.484629 32294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:42.484624 32295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:42.485324 32289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:42.486095 32289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:42.487285 32289 hybrid_clock.cc:648] HybridClock initialized: now 1786050402487271 us; error 29 us; skew 500 ppm
Aug 06 21:06:42 dist-test-slave-69rg krb5kdc[32175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050402, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:42.490160 32289 init.cc:377] Logged in from keytab as kudu/127.27.225.193@KRBTEST.COM (short username kudu)
I20260806 21:06:42.491060 32289 webserver.cc:460] Webserver started at http://127.27.225.193:36903/ using document root <none> and password file <none>
I20260806 21:06:42.491639 32289 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:42.491734 32289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:42.491993 32289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:42.494244 32289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "097f80d2efde4f64b8857233f99528bb"
format_stamp: "Formatted at 2026-08-06 21:06:42 on dist-test-slave-69rg"
server_key: "010679ad02b72e68ae4f558b0798daf3"
server_key_iv: "e3da46e30db1d5eec7e7217410248dc2"
server_key_version: "encryptionkey@0"
I20260806 21:06:42.494872 32289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "097f80d2efde4f64b8857233f99528bb"
format_stamp: "Formatted at 2026-08-06 21:06:42 on dist-test-slave-69rg"
server_key: "010679ad02b72e68ae4f558b0798daf3"
server_key_iv: "e3da46e30db1d5eec7e7217410248dc2"
server_key_version: "encryptionkey@0"
I20260806 21:06:42.499010 32289 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:06:42.502007 32304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:42.503371 32289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:06:42.503544 32289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "097f80d2efde4f64b8857233f99528bb"
format_stamp: "Formatted at 2026-08-06 21:06:42 on dist-test-slave-69rg"
server_key: "010679ad02b72e68ae4f558b0798daf3"
server_key_iv: "e3da46e30db1d5eec7e7217410248dc2"
server_key_version: "encryptionkey@0"
I20260806 21:06:42.503685 32289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:42.524943 32289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:42.528468 32289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:42.528697 32289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:42.529345 32289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:42.530382 32289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:42.530457 32289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:42.530527 32289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:42.530577 32289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:42.540045 32289 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:44745
I20260806 21:06:42.540064 32420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:44745 every 8 connection(s)
I20260806 21:06:42.541074 32289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:06:42.547960 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 32289
I20260806 21:06:42.548125 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:06:42.548470 28551 external_mini_cluster.cc:1428] Setting key 2b2c5387289d044284657fa12db2f0d9
Aug 06 21:06:42 dist-test-slave-69rg krb5kdc[32175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050402, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:42.557282 32265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:42.543161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:46989 (local address 127.27.225.254:41973)
0806 21:06:42.543359 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:42.543364 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:42.544718 (+ 1354us) server_negotiation.cc:407] Connection header received
0806 21:06:42.546490 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:42.546494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:42.546554 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:42.546643 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:42.548142 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:42.548953 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:42.550250 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:42.550492 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:42.550611 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:42.553215 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:42.553218 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:42.553220 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:42.553245 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:42.554999 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:42.555590 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:42.555594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:42.555596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:42.555692 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:42.556080 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:42.556084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:42.556086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:42.556295 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:42.556403 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:42.556949 (+ 546us) server_negotiation.cc:299] Negotiation successful
0806 21:06:42.557093 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 21:06:42.558195 32423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:42.543631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41973 (local address 127.27.225.193:46989)
0806 21:06:42.544477 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:42.544531 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:42.546214 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:42.546862 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:42.546870 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:42.547380 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:42.547936 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:42.547952 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:42.549107 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:42.549117 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:42.550115 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:42.550126 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:42.550662 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:42.550667 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:42.550845 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:42.551490 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:42.551508 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:42.553031 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:42.555142 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:42.555149 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:42.555168 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:42.555476 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:42.555805 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:42.555808 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:42.555810 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:42.555980 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:42.556418 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:42.556427 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:42.556690 (+ 263us) client_negotiation.cc:769] Sending connection context
0806 21:06:42.556892 (+ 202us) client_negotiation.cc:240] Negotiation successful
0806 21:06:42.557242 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":261,"threads_started":1}
WARNING: no policy specified for kudu/127.27.225.194@KRBTEST.COM; defaulting to no policy
I20260806 21:06:42.559873 32421 heartbeater.cc:347] Connected to a master server at 127.27.225.254:41973
I20260806 21:06:42.560139 32421 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:42.560758 32421 heartbeater.cc:511] Master 127.27.225.254:41973 requested a full tablet report, sending...
I20260806 21:06:42.562486 32224 ts_manager.cc:194] Registered new tserver with Master: 097f80d2efde4f64b8857233f99528bb (127.27.225.193:44745)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.194@KRBTEST.COM" created.
I20260806 21:06:42.563836 32224 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.193@KRBTEST.COM'} at 127.27.225.193:46989
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
I20260806 21:06:42.581084 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:41973
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:42.696211 32426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:42.696534 32426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:42.696583 32426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:42.700883 32426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:42.700960 32426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:42.701022 32426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:06:42.706640 32426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:41973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:42.707917 32426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:42.709009 32426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:42.718844 32434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:42.718892 32431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:42.718892 32432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:42.719211 32426 server_base.cc:1034] running on GCE node
I20260806 21:06:42.719655 32426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:42.720376 32426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:42.721585 32426 hybrid_clock.cc:648] HybridClock initialized: now 1786050402721552 us; error 51 us; skew 500 ppm
Aug 06 21:06:42 dist-test-slave-69rg krb5kdc[32175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050402, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:42.724646 32426 init.cc:377] Logged in from keytab as kudu/127.27.225.194@KRBTEST.COM (short username kudu)
I20260806 21:06:42.725520 32426 webserver.cc:460] Webserver started at http://127.27.225.194:43091/ using document root <none> and password file <none>
I20260806 21:06:42.726146 32426 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:42.726213 32426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:42.726388 32426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:42.728490 32426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "2720597e4e414a97a9300e8959dfdaba"
format_stamp: "Formatted at 2026-08-06 21:06:42 on dist-test-slave-69rg"
server_key: "3a631a505e86a4144ab4d568b6c39c31"
server_key_iv: "e5f2b6be1a9df7d64cf27fca896a77ee"
server_key_version: "encryptionkey@0"
I20260806 21:06:42.729101 32426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "2720597e4e414a97a9300e8959dfdaba"
format_stamp: "Formatted at 2026-08-06 21:06:42 on dist-test-slave-69rg"
server_key: "3a631a505e86a4144ab4d568b6c39c31"
server_key_iv: "e5f2b6be1a9df7d64cf27fca896a77ee"
server_key_version: "encryptionkey@0"
I20260806 21:06:42.733403 32426 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:06:42.736372 32441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:42.737727 32426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:06:42.737929 32426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "2720597e4e414a97a9300e8959dfdaba"
format_stamp: "Formatted at 2026-08-06 21:06:42 on dist-test-slave-69rg"
server_key: "3a631a505e86a4144ab4d568b6c39c31"
server_key_iv: "e5f2b6be1a9df7d64cf27fca896a77ee"
server_key_version: "encryptionkey@0"
I20260806 21:06:42.738087 32426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:42.769219 32426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:42.772863 32426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:42.773101 32426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:42.773753 32426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:42.774660 32426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:42.774735 32426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:42.774811 32426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:42.774864 32426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:42.784380 32426 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:40251
I20260806 21:06:42.784404 32560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:40251 every 8 connection(s)
I20260806 21:06:42.785331 32426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:06:42.789021 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 32426
I20260806 21:06:42.789129 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:06:42.789426 28551 external_mini_cluster.cc:1428] Setting key 1049307a74ac8e3e609eff429ce9b61b
Aug 06 21:06:42 dist-test-slave-69rg krb5kdc[32175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050402, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.225.195@KRBTEST.COM; defaulting to no policy
I20260806 21:06:42.800697 32265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:42.787319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:48731 (local address 127.27.225.254:41973)
0806 21:06:42.787466 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:42.787478 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:42.788256 (+ 778us) server_negotiation.cc:407] Connection header received
0806 21:06:42.789764 (+ 1508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:42.789767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:42.789864 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:42.789970 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:42.791373 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:42.791920 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:42.793243 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:42.793469 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:42.793557 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:42.796766 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:42.796769 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:42.796771 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:42.796798 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:42.798392 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:42.798986 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:42.798990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:42.798991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:42.799074 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:42.799460 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:42.799464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:42.799465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:42.799668 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:42.799779 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:42.800382 (+ 603us) server_negotiation.cc:299] Negotiation successful
0806 21:06:42.800521 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 21:06:42.801605 32563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:42.787599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41973 (local address 127.27.225.194:48731)
0806 21:06:42.788084 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:42.788132 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:42.789516 (+ 1384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:42.790176 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:42.790186 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:42.790709 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:42.791219 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:42.791233 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:42.792051 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:42.792054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:42.793086 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:42.793102 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:42.793642 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:42.793648 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:42.793924 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:42.794657 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:42.794684 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:42.796550 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:42.798531 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:42.798538 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:42.798556 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:42.798859 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:42.799176 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:42.799179 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:42.799181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:42.799349 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:42.799778 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:42.799785 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:42.800112 (+ 327us) client_negotiation.cc:769] Sending connection context
0806 21:06:42.800332 (+ 220us) client_negotiation.cc:240] Negotiation successful
0806 21:06:42.800689 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":115,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.195@KRBTEST.COM" created.
I20260806 21:06:42.802974 32561 heartbeater.cc:347] Connected to a master server at 127.27.225.254:41973
I20260806 21:06:42.803244 32561 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:42.803772 32561 heartbeater.cc:511] Master 127.27.225.254:41973 requested a full tablet report, sending...
I20260806 21:06:42.804908 32224 ts_manager.cc:194] Registered new tserver with Master: 2720597e4e414a97a9300e8959dfdaba (127.27.225.194:40251)
I20260806 21:06:42.805531 32224 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.194@KRBTEST.COM'} at 127.27.225.194:48731
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
I20260806 21:06:42.819317 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:41973
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:42.975821 32566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:42.976292 32566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:42.976370 32566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:42.982906 32566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:42.983048 32566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:42.983198 32566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:06:42.992017 32566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:41973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:42.993911 32566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:42.995575 32566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:43.011132 32571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:43.011778 32566 server_base.cc:1034] running on GCE node
W20260806 21:06:43.011185 32575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:43.011164 32572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:43.012668 32566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:43.013759 32566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:43.017215 32566 hybrid_clock.cc:648] HybridClock initialized: now 1786050403017145 us; error 69 us; skew 500 ppm
Aug 06 21:06:43 dist-test-slave-69rg krb5kdc[32175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050403, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:43.022653 32566 init.cc:377] Logged in from keytab as kudu/127.27.225.195@KRBTEST.COM (short username kudu)
I20260806 21:06:43.024155 32566 webserver.cc:460] Webserver started at http://127.27.225.195:44095/ using document root <none> and password file <none>
I20260806 21:06:43.025128 32566 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:43.025277 32566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:43.025650 32566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:43.029101 32566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "981f168395974248b090176b60757ef8"
format_stamp: "Formatted at 2026-08-06 21:06:43 on dist-test-slave-69rg"
server_key: "c3179e12d9ed94ccd3029690dc6059d0"
server_key_iv: "5532bc1c52dabbf0219be94e0d0ffe81"
server_key_version: "encryptionkey@0"
I20260806 21:06:43.030200 32566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "981f168395974248b090176b60757ef8"
format_stamp: "Formatted at 2026-08-06 21:06:43 on dist-test-slave-69rg"
server_key: "c3179e12d9ed94ccd3029690dc6059d0"
server_key_iv: "5532bc1c52dabbf0219be94e0d0ffe81"
server_key_version: "encryptionkey@0"
I20260806 21:06:43.036907 32566 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 21:06:43.041399 32582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:43.043694 32566 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 21:06:43.043864 32566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "981f168395974248b090176b60757ef8"
format_stamp: "Formatted at 2026-08-06 21:06:43 on dist-test-slave-69rg"
server_key: "c3179e12d9ed94ccd3029690dc6059d0"
server_key_iv: "5532bc1c52dabbf0219be94e0d0ffe81"
server_key_version: "encryptionkey@0"
I20260806 21:06:43.044009 32566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:43.063380 32566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:43.068835 32566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:43.069178 32566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:43.070165 32566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:43.071432 32566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:43.071511 32566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:43.071580 32566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:43.071614 32566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:43.087452 32566 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:35917
I20260806 21:06:43.087493 32698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:35917 every 8 connection(s)
I20260806 21:06:43.089042 32566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:06:43.089933 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 32566
I20260806 21:06:43.090133 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:06:43.090728 28551 external_mini_cluster.cc:1428] Setting key e93db438f3c7bee6f928bcbaf64a73fa
Aug 06 21:06:43 dist-test-slave-69rg krb5kdc[32175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050403, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:43.114588 32265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:43.092565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:38755 (local address 127.27.225.254:41973)
0806 21:06:43.092842 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:43.092846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:43.094808 (+ 1962us) server_negotiation.cc:407] Connection header received
0806 21:06:43.097247 (+ 2439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:43.097252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:43.097408 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:43.097600 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:43.099754 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:43.100507 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:43.102492 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:43.102807 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:43.102982 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:43.108029 (+ 5047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:43.108033 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:43.108035 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:43.108081 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:43.110733 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:43.111705 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:43.111709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:43.111710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:43.111849 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:43.112577 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:43.112581 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:43.112585 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:43.112916 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:43.113137 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:43.114096 (+ 959us) server_negotiation.cc:299] Negotiation successful
0806 21:06:43.114312 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260806 21:06:43.116081 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:43.093513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41973 (local address 127.27.225.195:38755)
0806 21:06:43.094555 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:43.094663 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:43.096856 (+ 2193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:43.097957 (+ 1101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:43.097967 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:43.098647 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:43.099494 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:43.099519 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:43.100752 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:43.100755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:43.102309 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:43.102320 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:43.103063 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:43.103069 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:43.103336 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:43.104271 (+ 935us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:43.104305 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:43.107706 (+ 3401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:43.110961 (+ 3255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:43.110977 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:43.111006 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:43.111492 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:43.112085 (+ 593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:43.112091 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:43.112101 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:43.112367 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:43.113130 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:43.113148 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:43.113614 (+ 466us) client_negotiation.cc:769] Sending connection context
0806 21:06:43.113994 (+ 380us) client_negotiation.cc:240] Negotiation successful
0806 21:06:43.114610 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":257,"threads_started":1}
I20260806 21:06:43.117786 32699 heartbeater.cc:347] Connected to a master server at 127.27.225.254:41973
I20260806 21:06:43.118193 32699 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:43.119091 32699 heartbeater.cc:511] Master 127.27.225.254:41973 requested a full tablet report, sending...
I20260806 21:06:43.120685 32224 ts_manager.cc:194] Registered new tserver with Master: 981f168395974248b090176b60757ef8 (127.27.225.195:35917)
I20260806 21:06:43.121547 32224 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.195@KRBTEST.COM'} at 127.27.225.195:38755
I20260806 21:06:43.133711 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 21:06:43 dist-test-slave-69rg krb5kdc[32175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 21:06:43 dist-test-slave-69rg krb5kdc[32175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.193@KRBTEST.COM
I20260806 21:06:43.178820 32705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:43.159689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50482 (local address 127.27.225.193:44745)
0806 21:06:43.160116 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:43.160137 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:43.160362 (+ 225us) server_negotiation.cc:407] Connection header received
0806 21:06:43.160553 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:43.160566 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:43.160909 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:43.161113 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:43.162341 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:43.163480 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:43.165057 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:43.165654 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:43.165793 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:43.170479 (+ 4686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:43.170492 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:43.170521 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:43.170566 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:43.174432 (+ 3866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:43.175578 (+ 1146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:43.175586 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:43.175588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:43.175758 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:43.176573 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:43.176579 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:43.176588 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:43.177137 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:43.177615 (+ 478us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:43.178158 (+ 543us) server_negotiation.cc:299] Negotiation successful
0806 21:06:43.178486 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":143,"threads_started":1}
Aug 06 21:06:43 dist-test-slave-69rg krb5kdc[32175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:43.200003 32265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:43.183389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53254 (local address 127.27.225.254:41973)
0806 21:06:43.183655 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:43.183665 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:43.183958 (+ 293us) server_negotiation.cc:407] Connection header received
0806 21:06:43.184211 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:43.184222 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:43.184316 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:43.184456 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:43.185494 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:43.186379 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:43.187741 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:43.188110 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:43.188366 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:43.192931 (+ 4565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:43.192935 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:43.192944 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:43.193006 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:43.195815 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:43.196876 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:43.196895 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:43.196908 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:43.197096 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:43.197806 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:43.197911 (+ 105us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:43.197912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:43.198394 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:43.198743 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:43.199306 (+ 563us) server_negotiation.cc:299] Negotiation successful
0806 21:06:43.199628 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
W20260806 21:06:43.200743 32224 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53254
I20260806 21:06:43.201934 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 32289
I20260806 21:06:43.212791 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 32426
I20260806 21:06:43.223176 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 32566
I20260806 21:06:43.233029 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 32189
2026-08-06T21:06:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2495 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:06:43 dist-test-slave-69rg krb5kdc[32711](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:06:43 dist-test-slave-69rg krb5kdc[32711](info): set up 2 sockets
Aug 06 21:06:43 dist-test-slave-69rg krb5kdc[32711](info): commencing operation
krb5kdc: starting...
W20260806 21:06:44.369730 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.002s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:06:44 dist-test-slave-69rg krb5kdc[32711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:06:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:06:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:06:44.505646 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:32865
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:33603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:32865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:44.620182 32724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:44.620604 32724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:44.620703 32724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:44.625149 32724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:06:44.625267 32724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:44.625308 32724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:44.625370 32724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:06:44.625411 32724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:06:44.631181 32724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:33603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:32865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:32865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:44.632398 32724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:44.633525 32724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:06:44.642365 32724 server_base.cc:1034] running on GCE node
W20260806 21:06:44.642575 32730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:44.642560 32732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:44.642601 32729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:44.643409 32724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:44.644538 32724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:44.645733 32724 hybrid_clock.cc:648] HybridClock initialized: now 1786050404645705 us; error 44 us; skew 500 ppm
Aug 06 21:06:44 dist-test-slave-69rg krb5kdc[32711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050404, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:44.648947 32724 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:06:44.649734 32724 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:44.649863 32724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:44.650079 32724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:44.652238 32724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "d7a0544b716345dd925e51ef17006686"
format_stamp: "Formatted at 2026-08-06 21:06:44 on dist-test-slave-69rg"
server_key: "e540595b2e7f7fdd57aded75900378b9"
server_key_iv: "3f8c7e1bb4c686bf52a8df37f1dbe762"
server_key_version: "encryptionkey@0"
I20260806 21:06:44.652853 32724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "d7a0544b716345dd925e51ef17006686"
format_stamp: "Formatted at 2026-08-06 21:06:44 on dist-test-slave-69rg"
server_key: "e540595b2e7f7fdd57aded75900378b9"
server_key_iv: "3f8c7e1bb4c686bf52a8df37f1dbe762"
server_key_version: "encryptionkey@0"
I20260806 21:06:44.656661 32724 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 21:06:44.659164 32738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:44.660604 32724 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 21:06:44.660746 32724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "d7a0544b716345dd925e51ef17006686"
format_stamp: "Formatted at 2026-08-06 21:06:44 on dist-test-slave-69rg"
server_key: "e540595b2e7f7fdd57aded75900378b9"
server_key_iv: "3f8c7e1bb4c686bf52a8df37f1dbe762"
server_key_version: "encryptionkey@0"
I20260806 21:06:44.660857 32724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:44.676031 32724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:44.679353 32724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:44.679571 32724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:44.687148 32724 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:32865
I20260806 21:06:44.687497 322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:32865 every 8 connection(s)
I20260806 21:06:44.687923 32724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:06:44.690953 323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:44.693575 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 32724
I20260806 21:06:44.693746 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:06:44.694089 28551 external_mini_cluster.cc:1428] Setting key cf6a7371045555f77d87c75fba295293
I20260806 21:06:44.697559 323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7a0544b716345dd925e51ef17006686: Bootstrap starting.
I20260806 21:06:44.699998 323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7a0544b716345dd925e51ef17006686: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:44.700784 323 log.cc:824] T 00000000000000000000000000000000 P d7a0544b716345dd925e51ef17006686: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:44.703001 323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7a0544b716345dd925e51ef17006686: No bootstrap required, opened a new log
Aug 06 21:06:44 dist-test-slave-69rg krb5kdc[32711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:44.706794 323 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7a0544b716345dd925e51ef17006686 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7a0544b716345dd925e51ef17006686" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 32865 } }
I20260806 21:06:44.707063 323 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7a0544b716345dd925e51ef17006686 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:44.707140 323 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7a0544b716345dd925e51ef17006686 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7a0544b716345dd925e51ef17006686, State: Initialized, Role: FOLLOWER
I20260806 21:06:44.707635 323 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7a0544b716345dd925e51ef17006686 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7a0544b716345dd925e51ef17006686" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 32865 } }
I20260806 21:06:44.707819 323 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7a0544b716345dd925e51ef17006686 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:06:44.707895 323 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7a0544b716345dd925e51ef17006686 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:06:44.708039 323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7a0544b716345dd925e51ef17006686 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:44.709249 323 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7a0544b716345dd925e51ef17006686 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7a0544b716345dd925e51ef17006686" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 32865 } }
I20260806 21:06:44.709589 323 leader_election.cc:302] T 00000000000000000000000000000000 P d7a0544b716345dd925e51ef17006686 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7a0544b716345dd925e51ef17006686; no voters:
I20260806 21:06:44.709983 323 leader_election.cc:288] T 00000000000000000000000000000000 P d7a0544b716345dd925e51ef17006686 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:06:44.710062 328 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7a0544b716345dd925e51ef17006686 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:44.710317 328 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7a0544b716345dd925e51ef17006686 [term 1 LEADER]: Becoming Leader. State: Replica: d7a0544b716345dd925e51ef17006686, State: Running, Role: LEADER
I20260806 21:06:44.710434 326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:44.695670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53522 (local address 127.27.225.254:32865)
0806 21:06:44.696204 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:44.696220 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:44.696259 (+ 39us) server_negotiation.cc:407] Connection header received
0806 21:06:44.696910 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:44.696927 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:44.697271 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:44.697648 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:44.698577 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:44.699623 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:44.700613 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:44.701164 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:44.701345 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:44.704250 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:44.704256 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:44.704269 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:44.704297 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:44.706934 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:44.707485 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:44.707492 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:44.707499 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:44.707630 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:44.707941 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:44.707944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:44.707946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:44.708449 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:44.708624 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:44.708943 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 21:06:44.709279 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":146,"threads_started":1}
I20260806 21:06:44.710812 328 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7a0544b716345dd925e51ef17006686 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7a0544b716345dd925e51ef17006686" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 32865 } }
I20260806 21:06:44.711331 323 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7a0544b716345dd925e51ef17006686 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:06:44.712046 328 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7a0544b716345dd925e51ef17006686 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7a0544b716345dd925e51ef17006686. Latest consensus state: current_term: 1 leader_uuid: "d7a0544b716345dd925e51ef17006686" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7a0544b716345dd925e51ef17006686" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 32865 } } }
I20260806 21:06:44.712270 328 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7a0544b716345dd925e51ef17006686 [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:44.712648 329 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7a0544b716345dd925e51ef17006686 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7a0544b716345dd925e51ef17006686" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7a0544b716345dd925e51ef17006686" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 32865 } } }
I20260806 21:06:44.712819 329 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7a0544b716345dd925e51ef17006686 [sys.catalog]: This master's current role is: LEADER
W20260806 21:06:44.716223 343 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d7a0544b716345dd925e51ef17006686: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:06:44.716292 343 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:06:44.716382 337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:06:44.717048 337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:06:44.723460 337 catalog_manager.cc:1347] Generated new cluster ID: 20ab0558a0cd4a638b7d02ad2139be65
I20260806 21:06:44.723548 337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:06:44.761121 337 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:06:44.762276 337 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:06:44.769651 337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7a0544b716345dd925e51ef17006686: Generated new TSK 0
I20260806 21:06:44.770538 337 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.225.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
I20260806 21:06:44.813318 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:32865
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:33603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:44.930073 349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:44.930394 349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:44.930473 349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:44.934648 349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:44.934746 349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:44.934840 349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:06:44.940366 349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:33603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:32865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:44.941468 349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:44.942618 349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:44.952163 358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:44.952227 356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:44.952255 355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:44.952455 349 server_base.cc:1034] running on GCE node
I20260806 21:06:44.952870 349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:44.953550 349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:44.954746 349 hybrid_clock.cc:648] HybridClock initialized: now 1786050404954734 us; error 60 us; skew 500 ppm
Aug 06 21:06:44 dist-test-slave-69rg krb5kdc[32711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050404, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:44.963685 349 init.cc:377] Logged in from keytab as kudu/127.27.225.193@KRBTEST.COM (short username kudu)
I20260806 21:06:44.964622 349 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:44.964722 349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:44.964965 349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:44.967209 349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "531c1f0027de4699b7ed511a4e01f555"
format_stamp: "Formatted at 2026-08-06 21:06:44 on dist-test-slave-69rg"
server_key: "ba8cc78bf671f3808435f30c7a0dead8"
server_key_iv: "de08c5e11e069a90d1aad6cdcd80fd8c"
server_key_version: "encryptionkey@0"
I20260806 21:06:44.967790 349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "531c1f0027de4699b7ed511a4e01f555"
format_stamp: "Formatted at 2026-08-06 21:06:44 on dist-test-slave-69rg"
server_key: "ba8cc78bf671f3808435f30c7a0dead8"
server_key_iv: "de08c5e11e069a90d1aad6cdcd80fd8c"
server_key_version: "encryptionkey@0"
I20260806 21:06:44.971540 349 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 21:06:44.974009 364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:44.975225 349 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:06:44.975364 349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "531c1f0027de4699b7ed511a4e01f555"
format_stamp: "Formatted at 2026-08-06 21:06:44 on dist-test-slave-69rg"
server_key: "ba8cc78bf671f3808435f30c7a0dead8"
server_key_iv: "de08c5e11e069a90d1aad6cdcd80fd8c"
server_key_version: "encryptionkey@0"
I20260806 21:06:44.975478 349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:44.996312 349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:44.999933 349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:45.000180 349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:45.000818 349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:45.001788 349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:45.001927 349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:45.002005 349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:45.002048 349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:45.011936 349 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:37813
I20260806 21:06:45.012001 478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:37813 every 8 connection(s)
I20260806 21:06:45.012629 349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:06:45.021742 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 349
I20260806 21:06:45.021915 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:06:45.022255 28551 external_mini_cluster.cc:1428] Setting key 90a6eda1dc5bd9aaae1fd9265027c0f2
Aug 06 21:06:45 dist-test-slave-69rg krb5kdc[32711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050404, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:45.027949 326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:45.014744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:58953 (local address 127.27.225.254:32865)
0806 21:06:45.014939 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:45.014943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:45.015819 (+ 876us) server_negotiation.cc:407] Connection header received
0806 21:06:45.017233 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:45.017238 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:45.017300 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:45.017373 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:45.018740 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.019294 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.020469 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.020722 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.020837 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:45.023752 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:45.023756 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:45.023759 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:45.023791 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:45.025351 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:45.026123 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:45.026128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:45.026131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:45.026239 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:45.026648 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:45.026655 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:45.026656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:45.026943 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:45.027057 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:45.027617 (+ 560us) server_negotiation.cc:299] Negotiation successful
0806 21:06:45.027775 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 21:06:45.028960 481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:45.015137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:32865 (local address 127.27.225.193:58953)
0806 21:06:45.015636 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:45.015686 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:45.017000 (+ 1314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:45.017548 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:45.017555 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:45.018027 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:45.018564 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:45.018579 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.019444 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.019453 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:45.020337 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:45.020347 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.020904 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.020909 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:45.021077 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:45.021728 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:45.021750 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:45.023586 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:45.025490 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:45.025500 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:45.025519 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:45.025976 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:45.026360 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:45.026362 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:45.026363 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:45.026510 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:45.027068 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:45.027075 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:45.027340 (+ 265us) client_negotiation.cc:769] Sending connection context
0806 21:06:45.027551 (+ 211us) client_negotiation.cc:240] Negotiation successful
0806 21:06:45.027950 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":131,"threads_started":1}
I20260806 21:06:45.030412 479 heartbeater.cc:347] Connected to a master server at 127.27.225.254:32865
I20260806 21:06:45.030690 479 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:45.031196 479 heartbeater.cc:511] Master 127.27.225.254:32865 requested a full tablet report, sending...
I20260806 21:06:45.032946 32755 ts_manager.cc:194] Registered new tserver with Master: 531c1f0027de4699b7ed511a4e01f555 (127.27.225.193:37813)
WARNING: no policy specified for kudu/127.27.225.194@KRBTEST.COM; defaulting to no policy
I20260806 21:06:45.034524 32755 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.193@KRBTEST.COM'} at 127.27.225.193:58953
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
I20260806 21:06:45.057119 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:32865
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:33603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:45.181299 484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:45.181602 484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:45.181650 484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:45.186813 484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:45.186900 484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:45.187006 484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:06:45.192761 484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:33603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:32865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:45.193898 484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:45.194931 484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:45.204686 493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:45.204713 490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:45.204814 484 server_base.cc:1034] running on GCE node
W20260806 21:06:45.204684 491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:45.205430 484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:45.206166 484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:45.207379 484 hybrid_clock.cc:648] HybridClock initialized: now 1786050405207359 us; error 29 us; skew 500 ppm
Aug 06 21:06:45 dist-test-slave-69rg krb5kdc[32711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050405, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:45.210630 484 init.cc:377] Logged in from keytab as kudu/127.27.225.194@KRBTEST.COM (short username kudu)
I20260806 21:06:45.211508 484 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:45.211615 484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:45.211856 484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:45.214399 484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "674a387eb35143289acc74cb3b1e05a1"
format_stamp: "Formatted at 2026-08-06 21:06:45 on dist-test-slave-69rg"
server_key: "cfc4bea9a67ca5a7921fa30bec998dee"
server_key_iv: "5f0c7f00f5e4f1699e8bcf6119d10022"
server_key_version: "encryptionkey@0"
I20260806 21:06:45.214987 484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "674a387eb35143289acc74cb3b1e05a1"
format_stamp: "Formatted at 2026-08-06 21:06:45 on dist-test-slave-69rg"
server_key: "cfc4bea9a67ca5a7921fa30bec998dee"
server_key_iv: "5f0c7f00f5e4f1699e8bcf6119d10022"
server_key_version: "encryptionkey@0"
I20260806 21:06:45.218940 484 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:06:45.221516 499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:45.222982 484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:06:45.223133 484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "674a387eb35143289acc74cb3b1e05a1"
format_stamp: "Formatted at 2026-08-06 21:06:45 on dist-test-slave-69rg"
server_key: "cfc4bea9a67ca5a7921fa30bec998dee"
server_key_iv: "5f0c7f00f5e4f1699e8bcf6119d10022"
server_key_version: "encryptionkey@0"
I20260806 21:06:45.223321 484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:45.238456 484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:45.241888 484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:45.242094 484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:45.242719 484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:45.243599 484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:45.243671 484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:45.243734 484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:45.243778 484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:45.252751 484 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:34631
I20260806 21:06:45.252779 612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:34631 every 8 connection(s)
I20260806 21:06:45.253430 484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:06:45.256425 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 484
I20260806 21:06:45.256546 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:06:45.256861 28551 external_mini_cluster.cc:1428] Setting key e5ee94838c568f8db8358921c6b3a7c4
Aug 06 21:06:45 dist-test-slave-69rg krb5kdc[32711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050405, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.225.195@KRBTEST.COM; defaulting to no policy
I20260806 21:06:45.268675 326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:45.255184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:49649 (local address 127.27.225.254:32865)
0806 21:06:45.255339 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:45.255343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:45.256200 (+ 857us) server_negotiation.cc:407] Connection header received
0806 21:06:45.257867 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:45.257871 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:45.257931 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:45.258034 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:45.259313 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.259853 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.261008 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.261244 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.261362 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:45.264754 (+ 3392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:45.264757 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:45.264760 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:45.264788 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:45.266506 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:45.267052 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:45.267056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:45.267058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:45.267153 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:45.267513 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:45.267518 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:45.267519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:45.267725 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:45.267838 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:45.268365 (+ 527us) server_negotiation.cc:299] Negotiation successful
0806 21:06:45.268506 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 21:06:45.269677 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:45.255559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:32865 (local address 127.27.225.194:49649)
0806 21:06:45.256027 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:45.256070 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:45.257612 (+ 1542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:45.258244 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:45.258253 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:45.258679 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:45.259160 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:45.259174 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.259983 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.259986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:45.260878 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:45.260888 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.261404 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.261408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:45.261580 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:45.262271 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:45.262296 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:45.264453 (+ 2157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:45.266635 (+ 2182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:45.266642 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:45.266658 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:45.266942 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:45.267252 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:45.267255 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:45.267257 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:45.267415 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:45.267847 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:45.267857 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:45.268120 (+ 263us) client_negotiation.cc:769] Sending connection context
0806 21:06:45.268326 (+ 206us) client_negotiation.cc:240] Negotiation successful
0806 21:06:45.268670 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":126,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.195@KRBTEST.COM" created.
I20260806 21:06:45.271103 613 heartbeater.cc:347] Connected to a master server at 127.27.225.254:32865
I20260806 21:06:45.271416 613 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:45.271896 613 heartbeater.cc:511] Master 127.27.225.254:32865 requested a full tablet report, sending...
I20260806 21:06:45.273056 32755 ts_manager.cc:194] Registered new tserver with Master: 674a387eb35143289acc74cb3b1e05a1 (127.27.225.194:34631)
I20260806 21:06:45.273684 32755 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.194@KRBTEST.COM'} at 127.27.225.194:49649
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
I20260806 21:06:45.290949 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:32865
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:33603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:45.407285 618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:45.407630 618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:45.407711 618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:45.412129 618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:45.412230 618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:45.412333 618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:06:45.418043 618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:33603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:32865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:45.419229 618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:45.420308 618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:45.430289 625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:45.430423 627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:45.430289 624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:45.430507 618 server_base.cc:1034] running on GCE node
I20260806 21:06:45.431121 618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:45.431893 618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:45.433099 618 hybrid_clock.cc:648] HybridClock initialized: now 1786050405433078 us; error 39 us; skew 500 ppm
Aug 06 21:06:45 dist-test-slave-69rg krb5kdc[32711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050405, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:45.436344 618 init.cc:377] Logged in from keytab as kudu/127.27.225.195@KRBTEST.COM (short username kudu)
I20260806 21:06:45.437184 618 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:45.437283 618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:45.437525 618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:45.439826 618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "5f5ed7c4937e46f9b1a196660738270a"
format_stamp: "Formatted at 2026-08-06 21:06:45 on dist-test-slave-69rg"
server_key: "b45f05fbbdecbf0ba33bbe5d936f9f3a"
server_key_iv: "fe7fa8deccc5be6ff28a9c91021ad2dc"
server_key_version: "encryptionkey@0"
I20260806 21:06:45.440418 618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "5f5ed7c4937e46f9b1a196660738270a"
format_stamp: "Formatted at 2026-08-06 21:06:45 on dist-test-slave-69rg"
server_key: "b45f05fbbdecbf0ba33bbe5d936f9f3a"
server_key_iv: "fe7fa8deccc5be6ff28a9c91021ad2dc"
server_key_version: "encryptionkey@0"
I20260806 21:06:45.444401 618 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 21:06:45.446967 633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:45.448336 618 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 21:06:45.448487 618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "5f5ed7c4937e46f9b1a196660738270a"
format_stamp: "Formatted at 2026-08-06 21:06:45 on dist-test-slave-69rg"
server_key: "b45f05fbbdecbf0ba33bbe5d936f9f3a"
server_key_iv: "fe7fa8deccc5be6ff28a9c91021ad2dc"
server_key_version: "encryptionkey@0"
I20260806 21:06:45.448611 618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:45.462294 618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:45.465659 618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:45.465966 618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:45.466537 618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:45.467363 618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:45.467412 618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:45.467451 618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:45.467468 618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:45.476317 618 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:43971
I20260806 21:06:45.476341 747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:43971 every 8 connection(s)
I20260806 21:06:45.476922 618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:06:45.478837 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 618
I20260806 21:06:45.478961 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:06:45.479278 28551 external_mini_cluster.cc:1428] Setting key 9e752fd197c6952189119477b945b510
Aug 06 21:06:45 dist-test-slave-69rg krb5kdc[32711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050405, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:45.491225 326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:45.478796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:58477 (local address 127.27.225.254:32865)
0806 21:06:45.478945 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:45.478951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:45.480023 (+ 1072us) server_negotiation.cc:407] Connection header received
0806 21:06:45.481916 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:45.481920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:45.481975 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:45.482056 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:45.483220 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.483769 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.484904 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.485140 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.485253 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:45.487785 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:45.487789 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:45.487791 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:45.487826 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:45.489091 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:45.489623 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:45.489627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:45.489628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:45.489721 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:45.490104 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:45.490108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:45.490110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:45.490305 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:45.490417 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:45.490914 (+ 497us) server_negotiation.cc:299] Negotiation successful
0806 21:06:45.491045 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 21:06:45.492076 750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:45.479314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:32865 (local address 127.27.225.195:58477)
0806 21:06:45.479877 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:45.479920 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:45.481668 (+ 1748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:45.482239 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:45.482246 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:45.482601 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:45.483069 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:45.483082 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.483898 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.483901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:45.484776 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:45.484785 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.485292 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.485296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:45.485474 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:45.486160 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:45.486176 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:45.487611 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:45.489239 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:45.489245 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:45.489265 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:45.489514 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:45.489849 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:45.489852 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:45.489854 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:45.489996 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:45.490416 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:45.490423 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:45.490672 (+ 249us) client_negotiation.cc:769] Sending connection context
0806 21:06:45.490857 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 21:06:45.491196 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":154,"threads_started":1}
I20260806 21:06:45.493373 748 heartbeater.cc:347] Connected to a master server at 127.27.225.254:32865
I20260806 21:06:45.493647 748 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:45.494161 748 heartbeater.cc:511] Master 127.27.225.254:32865 requested a full tablet report, sending...
I20260806 21:06:45.495225 32755 ts_manager.cc:194] Registered new tserver with Master: 5f5ed7c4937e46f9b1a196660738270a (127.27.225.195:43971)
I20260806 21:06:45.495812 32755 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.195@KRBTEST.COM'} at 127.27.225.195:58477
I20260806 21:06:45.505996 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:06:45.516741 326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:45.508179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53532 (local address 127.27.225.254:32865)
0806 21:06:45.508343 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:45.508348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:45.508526 (+ 178us) server_negotiation.cc:407] Connection header received
0806 21:06:45.508647 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:45.508650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:45.508695 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:45.508766 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:45.509554 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.510190 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.511028 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.511366 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.511459 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:45.513058 (+ 1599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:45.513062 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:45.513064 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:45.513091 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:45.514730 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:45.515254 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:45.515257 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:45.515259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:45.515360 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:45.515704 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:45.515707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:45.515708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:45.515957 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:45.516064 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:45.516374 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 21:06:45.516540 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 21:06:45.519994 32755 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53532:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:06:45.523201 32755 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:06:45.538817 763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:45.533090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:37813 (local address 127.0.0.1:53136)
0806 21:06:45.534051 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:45.534064 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:45.534135 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:45.534594 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:45.534599 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:45.534621 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:45.534821 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:45.534829 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.535868 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.535874 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:45.537089 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:45.537100 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.538200 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.538203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:45.538312 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:45.538349 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 21:06:45.538493 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 21:06:45.538631 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":905,"mutex_wait_us":58,"thread_start_us":99,"threads_started":1}
I20260806 21:06:45.539183 762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:45.532858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57674 (local address 127.27.225.194:34631)
0806 21:06:45.533301 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:45.533314 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:45.533687 (+ 373us) server_negotiation.cc:407] Connection header received
0806 21:06:45.533989 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:45.533995 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:45.534220 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:45.534377 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:45.534923 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.536135 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.537517 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.538372 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.538514 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:45.538649 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:45.538897 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 21:06:45.539022 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":120,"threads_started":1}
I20260806 21:06:45.539162 761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:45.532811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:43971 (local address 127.0.0.1:35006)
0806 21:06:45.533577 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:45.533593 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:45.533758 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:45.534956 (+ 1198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:45.534960 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:45.534982 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:45.535068 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:45.535074 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.536453 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.536457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:45.537551 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:45.537560 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.538730 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.538734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:45.538838 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:45.538867 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 21:06:45.538967 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 21:06:45.539039 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":105,"threads_started":1}
I20260806 21:06:45.539166 760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:45.532736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:34631 (local address 127.0.0.1:57674)
0806 21:06:45.533536 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:45.533592 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:45.533758 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:45.534576 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:45.534582 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:45.534606 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:45.534787 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:45.534800 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.536289 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.536292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:45.537379 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:45.537391 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.538507 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.538510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:45.538612 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:45.538633 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 21:06:45.538720 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 21:06:45.538780 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":111,"threads_started":1}
I20260806 21:06:45.539680 765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:45.533590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53136 (local address 127.27.225.193:37813)
0806 21:06:45.534107 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:45.534116 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:45.534140 (+ 24us) server_negotiation.cc:407] Connection header received
0806 21:06:45.534247 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:45.534252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:45.534436 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:45.534773 (+ 337us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:45.534917 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.535749 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.537235 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.538054 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.538209 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:45.538352 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:45.539399 (+ 1047us) server_negotiation.cc:299] Negotiation successful
0806 21:06:45.539518 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":114,"threads_started":1}
I20260806 21:06:45.539702 764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:45.533590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35006 (local address 127.27.225.195:43971)
0806 21:06:45.534449 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:45.534460 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:45.534481 (+ 21us) server_negotiation.cc:407] Connection header received
0806 21:06:45.534559 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:45.534564 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:45.534734 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:45.534888 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:45.535210 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.536303 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.537678 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.538528 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.538641 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:45.538768 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:45.539420 (+ 652us) server_negotiation.cc:299] Negotiation successful
0806 21:06:45.539541 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":755,"thread_start_us":114,"threads_started":1}
I20260806 21:06:45.542452 547 tablet_service.cc:1467] Processing CreateTablet for tablet d275b3c036914edc8a7e227624934488 (DEFAULT_TABLE table=test-table [id=cca51fbf7df74cb28f210f3c9086d95d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:45.542470 681 tablet_service.cc:1467] Processing CreateTablet for tablet d275b3c036914edc8a7e227624934488 (DEFAULT_TABLE table=test-table [id=cca51fbf7df74cb28f210f3c9086d95d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:45.542471 413 tablet_service.cc:1467] Processing CreateTablet for tablet d275b3c036914edc8a7e227624934488 (DEFAULT_TABLE table=test-table [id=cca51fbf7df74cb28f210f3c9086d95d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:45.543509 547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d275b3c036914edc8a7e227624934488. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:45.543533 413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d275b3c036914edc8a7e227624934488. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:45.543538 681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d275b3c036914edc8a7e227624934488. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:45.550386 766 tablet_bootstrap.cc:492] T d275b3c036914edc8a7e227624934488 P 674a387eb35143289acc74cb3b1e05a1: Bootstrap starting.
I20260806 21:06:45.551620 768 tablet_bootstrap.cc:492] T d275b3c036914edc8a7e227624934488 P 5f5ed7c4937e46f9b1a196660738270a: Bootstrap starting.
I20260806 21:06:45.552721 766 tablet_bootstrap.cc:654] T d275b3c036914edc8a7e227624934488 P 674a387eb35143289acc74cb3b1e05a1: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:45.553507 768 tablet_bootstrap.cc:654] T d275b3c036914edc8a7e227624934488 P 5f5ed7c4937e46f9b1a196660738270a: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:45.553978 766 log.cc:824] T d275b3c036914edc8a7e227624934488 P 674a387eb35143289acc74cb3b1e05a1: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:45.554234 767 tablet_bootstrap.cc:492] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555: Bootstrap starting.
I20260806 21:06:45.554809 768 log.cc:824] T d275b3c036914edc8a7e227624934488 P 5f5ed7c4937e46f9b1a196660738270a: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:45.556208 766 tablet_bootstrap.cc:492] T d275b3c036914edc8a7e227624934488 P 674a387eb35143289acc74cb3b1e05a1: No bootstrap required, opened a new log
I20260806 21:06:45.556345 767 tablet_bootstrap.cc:654] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:45.556414 766 ts_tablet_manager.cc:1397] T d275b3c036914edc8a7e227624934488 P 674a387eb35143289acc74cb3b1e05a1: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 21:06:45.556875 768 tablet_bootstrap.cc:492] T d275b3c036914edc8a7e227624934488 P 5f5ed7c4937e46f9b1a196660738270a: No bootstrap required, opened a new log
I20260806 21:06:45.557054 768 ts_tablet_manager.cc:1397] T d275b3c036914edc8a7e227624934488 P 5f5ed7c4937e46f9b1a196660738270a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:06:45.557118 767 log.cc:824] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:45.558899 767 tablet_bootstrap.cc:492] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555: No bootstrap required, opened a new log
I20260806 21:06:45.559101 767 ts_tablet_manager.cc:1397] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 21:06:45.560500 766 raft_consensus.cc:348] T d275b3c036914edc8a7e227624934488 P 674a387eb35143289acc74cb3b1e05a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "674a387eb35143289acc74cb3b1e05a1" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 34631 } } peers { permanent_uuid: "5f5ed7c4937e46f9b1a196660738270a" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43971 } } peers { permanent_uuid: "531c1f0027de4699b7ed511a4e01f555" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37813 } }
I20260806 21:06:45.560797 766 raft_consensus.cc:374] T d275b3c036914edc8a7e227624934488 P 674a387eb35143289acc74cb3b1e05a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:45.560846 766 raft_consensus.cc:729] T d275b3c036914edc8a7e227624934488 P 674a387eb35143289acc74cb3b1e05a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 674a387eb35143289acc74cb3b1e05a1, State: Initialized, Role: FOLLOWER
I20260806 21:06:45.560747 768 raft_consensus.cc:348] T d275b3c036914edc8a7e227624934488 P 5f5ed7c4937e46f9b1a196660738270a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "674a387eb35143289acc74cb3b1e05a1" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 34631 } } peers { permanent_uuid: "5f5ed7c4937e46f9b1a196660738270a" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43971 } } peers { permanent_uuid: "531c1f0027de4699b7ed511a4e01f555" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37813 } }
I20260806 21:06:45.561033 768 raft_consensus.cc:374] T d275b3c036914edc8a7e227624934488 P 5f5ed7c4937e46f9b1a196660738270a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:45.561097 768 raft_consensus.cc:729] T d275b3c036914edc8a7e227624934488 P 5f5ed7c4937e46f9b1a196660738270a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f5ed7c4937e46f9b1a196660738270a, State: Initialized, Role: FOLLOWER
I20260806 21:06:45.561358 766 consensus_queue.cc:260] T d275b3c036914edc8a7e227624934488 P 674a387eb35143289acc74cb3b1e05a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "674a387eb35143289acc74cb3b1e05a1" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 34631 } } peers { permanent_uuid: "5f5ed7c4937e46f9b1a196660738270a" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43971 } } peers { permanent_uuid: "531c1f0027de4699b7ed511a4e01f555" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37813 } }
I20260806 21:06:45.561470 768 consensus_queue.cc:260] T d275b3c036914edc8a7e227624934488 P 5f5ed7c4937e46f9b1a196660738270a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "674a387eb35143289acc74cb3b1e05a1" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 34631 } } peers { permanent_uuid: "5f5ed7c4937e46f9b1a196660738270a" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43971 } } peers { permanent_uuid: "531c1f0027de4699b7ed511a4e01f555" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37813 } }
I20260806 21:06:45.562515 766 ts_tablet_manager.cc:1428] T d275b3c036914edc8a7e227624934488 P 674a387eb35143289acc74cb3b1e05a1: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 21:06:45.562515 768 ts_tablet_manager.cc:1428] T d275b3c036914edc8a7e227624934488 P 5f5ed7c4937e46f9b1a196660738270a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 21:06:45.562893 748 heartbeater.cc:503] Master 127.27.225.254:32865 was elected leader, sending a full tablet report...
I20260806 21:06:45.562896 613 heartbeater.cc:503] Master 127.27.225.254:32865 was elected leader, sending a full tablet report...
I20260806 21:06:45.563313 767 raft_consensus.cc:348] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "674a387eb35143289acc74cb3b1e05a1" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 34631 } } peers { permanent_uuid: "5f5ed7c4937e46f9b1a196660738270a" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43971 } } peers { permanent_uuid: "531c1f0027de4699b7ed511a4e01f555" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37813 } }
I20260806 21:06:45.563558 767 raft_consensus.cc:374] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:45.563616 767 raft_consensus.cc:729] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 531c1f0027de4699b7ed511a4e01f555, State: Initialized, Role: FOLLOWER
I20260806 21:06:45.564030 767 consensus_queue.cc:260] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "674a387eb35143289acc74cb3b1e05a1" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 34631 } } peers { permanent_uuid: "5f5ed7c4937e46f9b1a196660738270a" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43971 } } peers { permanent_uuid: "531c1f0027de4699b7ed511a4e01f555" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37813 } }
I20260806 21:06:45.565080 767 ts_tablet_manager.cc:1428] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 21:06:45.565088 479 heartbeater.cc:503] Master 127.27.225.254:32865 was elected leader, sending a full tablet report...
I20260806 21:06:45.610750 775 raft_consensus.cc:482] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:06:45.611016 775 raft_consensus.cc:504] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "674a387eb35143289acc74cb3b1e05a1" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 34631 } } peers { permanent_uuid: "5f5ed7c4937e46f9b1a196660738270a" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43971 } } peers { permanent_uuid: "531c1f0027de4699b7ed511a4e01f555" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37813 } }
I20260806 21:06:45.612196 775 leader_election.cc:288] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 674a387eb35143289acc74cb3b1e05a1 (127.27.225.194:34631), 5f5ed7c4937e46f9b1a196660738270a (127.27.225.195:43971)
I20260806 21:06:45.616686 762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:45.612780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:35823 (local address 127.27.225.194:34631)
0806 21:06:45.612940 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:45.612947 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:45.613106 (+ 159us) server_negotiation.cc:407] Connection header received
0806 21:06:45.613267 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:45.613269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:45.613310 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:45.613397 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:45.613736 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.614426 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.615667 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.616071 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.616212 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:45.616269 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:45.616494 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 21:06:45.616560 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 21:06:45.616700 764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:45.612849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:60309 (local address 127.27.225.195:43971)
0806 21:06:45.612980 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:45.612986 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:45.613152 (+ 166us) server_negotiation.cc:407] Connection header received
0806 21:06:45.613313 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:45.613316 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:45.613366 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:45.613450 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:45.613850 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.614439 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.615653 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.616081 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.616225 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:45.616280 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:45.616514 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 21:06:45.616581 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 21:06:45.616748 777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:45.612462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:34631 (local address 127.27.225.193:35823)
0806 21:06:45.613010 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:45.613024 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:45.613155 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:45.613412 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:45.613414 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:45.613430 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:45.613610 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:45.613617 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.614544 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.614549 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:45.615535 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:45.615544 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.616210 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.616214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:45.616366 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:45.616385 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 21:06:45.616472 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 21:06:45.616547 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":96,"spinlock_wait_cycles":22912,"thread_start_us":242,"threads_started":1}
I20260806 21:06:45.616751 776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:45.612483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:43971 (local address 127.27.225.193:60309)
0806 21:06:45.613057 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:45.613068 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:45.613195 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:45.613461 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:45.613464 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:45.613504 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:45.613631 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:45.613636 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.614559 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.614562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:45.615526 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:45.615535 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.616211 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.616215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:45.616378 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:45.616396 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 21:06:45.616496 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 21:06:45.616567 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":152,"spinlock_wait_cycles":3584,"thread_start_us":234,"threads_started":1}
I20260806 21:06:45.617419 567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d275b3c036914edc8a7e227624934488" candidate_uuid: "531c1f0027de4699b7ed511a4e01f555" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "674a387eb35143289acc74cb3b1e05a1" is_pre_election: true
I20260806 21:06:45.617637 702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d275b3c036914edc8a7e227624934488" candidate_uuid: "531c1f0027de4699b7ed511a4e01f555" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f5ed7c4937e46f9b1a196660738270a" is_pre_election: true
I20260806 21:06:45.617726 567 raft_consensus.cc:2454] T d275b3c036914edc8a7e227624934488 P 674a387eb35143289acc74cb3b1e05a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 531c1f0027de4699b7ed511a4e01f555 in term 0.
I20260806 21:06:45.617930 702 raft_consensus.cc:2454] T d275b3c036914edc8a7e227624934488 P 5f5ed7c4937e46f9b1a196660738270a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 531c1f0027de4699b7ed511a4e01f555 in term 0.
I20260806 21:06:45.618283 366 leader_election.cc:302] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 531c1f0027de4699b7ed511a4e01f555, 674a387eb35143289acc74cb3b1e05a1; no voters:
I20260806 21:06:45.618574 775 raft_consensus.cc:2781] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:06:45.618662 775 raft_consensus.cc:482] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:06:45.618722 775 raft_consensus.cc:3037] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:45.619781 775 raft_consensus.cc:504] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "674a387eb35143289acc74cb3b1e05a1" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 34631 } } peers { permanent_uuid: "5f5ed7c4937e46f9b1a196660738270a" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43971 } } peers { permanent_uuid: "531c1f0027de4699b7ed511a4e01f555" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37813 } }
I20260806 21:06:45.620153 775 leader_election.cc:288] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555 [CANDIDATE]: Term 1 election: Requested vote from peers 674a387eb35143289acc74cb3b1e05a1 (127.27.225.194:34631), 5f5ed7c4937e46f9b1a196660738270a (127.27.225.195:43971)
I20260806 21:06:45.620483 702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d275b3c036914edc8a7e227624934488" candidate_uuid: "531c1f0027de4699b7ed511a4e01f555" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f5ed7c4937e46f9b1a196660738270a"
I20260806 21:06:45.620499 567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d275b3c036914edc8a7e227624934488" candidate_uuid: "531c1f0027de4699b7ed511a4e01f555" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "674a387eb35143289acc74cb3b1e05a1"
I20260806 21:06:45.620589 702 raft_consensus.cc:3037] T d275b3c036914edc8a7e227624934488 P 5f5ed7c4937e46f9b1a196660738270a [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:45.620630 567 raft_consensus.cc:3037] T d275b3c036914edc8a7e227624934488 P 674a387eb35143289acc74cb3b1e05a1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:45.621707 702 raft_consensus.cc:2454] T d275b3c036914edc8a7e227624934488 P 5f5ed7c4937e46f9b1a196660738270a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 531c1f0027de4699b7ed511a4e01f555 in term 1.
I20260806 21:06:45.621738 567 raft_consensus.cc:2454] T d275b3c036914edc8a7e227624934488 P 674a387eb35143289acc74cb3b1e05a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 531c1f0027de4699b7ed511a4e01f555 in term 1.
I20260806 21:06:45.622064 365 leader_election.cc:302] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 531c1f0027de4699b7ed511a4e01f555, 5f5ed7c4937e46f9b1a196660738270a; no voters:
I20260806 21:06:45.622324 775 raft_consensus.cc:2781] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:45.622589 775 raft_consensus.cc:686] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555 [term 1 LEADER]: Becoming Leader. State: Replica: 531c1f0027de4699b7ed511a4e01f555, State: Running, Role: LEADER
I20260806 21:06:45.622951 775 consensus_queue.cc:237] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "674a387eb35143289acc74cb3b1e05a1" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 34631 } } peers { permanent_uuid: "5f5ed7c4937e46f9b1a196660738270a" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43971 } } peers { permanent_uuid: "531c1f0027de4699b7ed511a4e01f555" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37813 } }
I20260806 21:06:45.626154 32755 catalog_manager.cc:5495] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555 reported cstate change: term changed from 0 to 1, leader changed from <none> to 531c1f0027de4699b7ed511a4e01f555 (127.27.225.193). New cstate: current_term: 1 leader_uuid: "531c1f0027de4699b7ed511a4e01f555" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "674a387eb35143289acc74cb3b1e05a1" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 34631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f5ed7c4937e46f9b1a196660738270a" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "531c1f0027de4699b7ed511a4e01f555" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37813 } health_report { overall_health: HEALTHY } } }
I20260806 21:06:45.638433 765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:45.634856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53146 (local address 127.27.225.193:37813)
0806 21:06:45.634987 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:45.634991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:45.635005 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:06:45.635157 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:45.635159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:45.635212 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:45.635300 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:06:45.635534 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.636040 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.636901 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:45.637146 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:45.637225 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:45.637490 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:06:45.637970 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:06:45.638119 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:45.638198 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 21:06:45.638250 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 21:06:45.644384 702 raft_consensus.cc:1261] T d275b3c036914edc8a7e227624934488 P 5f5ed7c4937e46f9b1a196660738270a [term 1 FOLLOWER]: Refusing update from remote peer 531c1f0027de4699b7ed511a4e01f555: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:06:45.644384 567 raft_consensus.cc:1261] T d275b3c036914edc8a7e227624934488 P 674a387eb35143289acc74cb3b1e05a1 [term 1 FOLLOWER]: Refusing update from remote peer 531c1f0027de4699b7ed511a4e01f555: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:06:45.645159 778 consensus_queue.cc:1035] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f5ed7c4937e46f9b1a196660738270a" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:06:45.645358 775 consensus_queue.cc:1035] T d275b3c036914edc8a7e227624934488 P 531c1f0027de4699b7ed511a4e01f555 [LEADER]: Connected to new peer: Peer: permanent_uuid: "674a387eb35143289acc74cb3b1e05a1" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 34631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:06:45.663071 32755 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:06:45.663287 32755 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:06:45.666100 32755 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d7a0544b716345dd925e51ef17006686: Sending DeleteTablet for 3 replicas of tablet d275b3c036914edc8a7e227624934488
I20260806 21:06:45.666972 547 tablet_service.cc:1514] Processing DeleteTablet for tablet d275b3c036914edc8a7e227624934488 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:45 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:57674
I20260806 21:06:45.666970 413 tablet_service.cc:1514] Processing DeleteTablet for tablet d275b3c036914edc8a7e227624934488 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:45 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:53136
I20260806 21:06:45.667053 681 tablet_service.cc:1514] Processing DeleteTablet for tablet d275b3c036914edc8a7e227624934488 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:45 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:35006
I20260806 21:06:45.667719 793 tablet_replica.cc:330] T d275b3c036914edc8a7e227624934488 P 5f5ed7c4937e46f9b1a196660738270a: stopping tablet replica
I20260806 21:06:45.668133 793 raft_consensus.cc:2229] T d275b3c036914edc8a7e227624934488 P 5f5ed7c4937e46f9b1a196660738270a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:06:45.668366 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 349
I20260806 21:06:45.668502 793 raft_consensus.cc:2258] T d275b3c036914edc8a7e227624934488 P 5f5ed7c4937e46f9b1a196660738270a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:45.670758 792 tablet_replica.cc:330] T d275b3c036914edc8a7e227624934488 P 674a387eb35143289acc74cb3b1e05a1: stopping tablet replica
I20260806 21:06:45.671120 792 raft_consensus.cc:2229] T d275b3c036914edc8a7e227624934488 P 674a387eb35143289acc74cb3b1e05a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:06:45.671458 792 raft_consensus.cc:2258] T d275b3c036914edc8a7e227624934488 P 674a387eb35143289acc74cb3b1e05a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:45.671548 793 ts_tablet_manager.cc:1905] T d275b3c036914edc8a7e227624934488 P 5f5ed7c4937e46f9b1a196660738270a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:45.672935 792 ts_tablet_manager.cc:1905] T d275b3c036914edc8a7e227624934488 P 674a387eb35143289acc74cb3b1e05a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:45.674571 793 ts_tablet_manager.cc:1918] T d275b3c036914edc8a7e227624934488 P 5f5ed7c4937e46f9b1a196660738270a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:06:45.674690 793 log.cc:1196] T d275b3c036914edc8a7e227624934488 P 5f5ed7c4937e46f9b1a196660738270a: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/d275b3c036914edc8a7e227624934488
I20260806 21:06:45.675101 793 ts_tablet_manager.cc:1939] T d275b3c036914edc8a7e227624934488 P 5f5ed7c4937e46f9b1a196660738270a: Deleting consensus metadata
I20260806 21:06:45.675805 792 ts_tablet_manager.cc:1918] T d275b3c036914edc8a7e227624934488 P 674a387eb35143289acc74cb3b1e05a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:06:45.675943 792 log.cc:1196] T d275b3c036914edc8a7e227624934488 P 674a387eb35143289acc74cb3b1e05a1: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050378084738-28551-0/minicluster-data/ts-1/wal/wals/d275b3c036914edc8a7e227624934488
I20260806 21:06:45.676302 792 ts_tablet_manager.cc:1939] T d275b3c036914edc8a7e227624934488 P 674a387eb35143289acc74cb3b1e05a1: Deleting consensus metadata
W20260806 21:06:45.676415 32742 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.225.193:37813 (error 108)
I20260806 21:06:45.676559 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 484
I20260806 21:06:45.676571 32739 catalog_manager.cc:4841] TS 5f5ed7c4937e46f9b1a196660738270a (127.27.225.195:43971): tablet d275b3c036914edc8a7e227624934488 (table test-table [id=cca51fbf7df74cb28f210f3c9086d95d]) successfully deleted
I20260806 21:06:45.678987 760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:45.678702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:37813 (local address 127.0.0.1:53158)
0806 21:06:45.678839 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:45.678911 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.193:37813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 21:06:45.679196 32742 catalog_manager.cc:4584] TS 531c1f0027de4699b7ed511a4e01f555 (127.27.225.193:37813): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d275b3c036914edc8a7e227624934488: Network error: Client connection negotiation failed: client connection to 127.27.225.193:37813: connect: Connection refused (error 111)
I20260806 21:06:45.682168 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 618
I20260806 21:06:45.683830 760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:45.682304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:34631 (local address 127.0.0.1:57686)
0806 21:06:45.683686 (+ 1382us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:45.683745 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.194:34631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1298}
I20260806 21:06:45.687801 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 32724
2026-08-06T21:06:45Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2445 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T21:06:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:06:45Z Disabled control of system clock
I20260806 21:06:45.721379 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:40121
--webserver_interface=127.27.225.254
--webserver_port=0
--builtin_ntp_servers=127.27.225.212:37139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:40121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 21:06:45.845278 798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:45.845618 798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:45.845660 798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:45.850041 798 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:06:45.850128 798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:45.850144 798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:45.850163 798 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:06:45.850179 798 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:06:45.855381 798 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:37139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:40121
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:40121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:45.856580 798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:45.858049 798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:45.867805 804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:45.867796 803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:45.867775 806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:45.868407 798 server_base.cc:1034] running on GCE node
I20260806 21:06:45.869063 798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:45.870425 798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:45.871626 798 hybrid_clock.cc:648] HybridClock initialized: now 1786050405871599 us; error 42 us; skew 500 ppm
I20260806 21:06:45.874334 798 webserver.cc:460] Webserver started at http://127.27.225.254:38649/ using document root <none> and password file <none>
I20260806 21:06:45.874991 798 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:45.875065 798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:45.875334 798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:45.878198 798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "f29d4d0e24804518a1d835a29200eb60"
format_stamp: "Formatted at 2026-08-06 21:06:45 on dist-test-slave-69rg"
server_key: "d306e2242b1b7c3cbdd87676be529c9e"
server_key_iv: "2545a4987f5044b195b9d375ad955e56"
server_key_version: "encryptionkey@0"
I20260806 21:06:45.879036 798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "f29d4d0e24804518a1d835a29200eb60"
format_stamp: "Formatted at 2026-08-06 21:06:45 on dist-test-slave-69rg"
server_key: "d306e2242b1b7c3cbdd87676be529c9e"
server_key_iv: "2545a4987f5044b195b9d375ad955e56"
server_key_version: "encryptionkey@0"
I20260806 21:06:45.884407 798 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 21:06:45.887578 812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:45.889652 798 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:06:45.889902 798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "f29d4d0e24804518a1d835a29200eb60"
format_stamp: "Formatted at 2026-08-06 21:06:45 on dist-test-slave-69rg"
server_key: "d306e2242b1b7c3cbdd87676be529c9e"
server_key_iv: "2545a4987f5044b195b9d375ad955e56"
server_key_version: "encryptionkey@0"
I20260806 21:06:45.890069 798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:45.895202 798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:45.895929 798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:45.896159 798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:45.907629 798 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:40121
I20260806 21:06:45.907629 865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:40121 every 8 connection(s)
I20260806 21:06:45.908977 798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:06:45.909323 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 798
I20260806 21:06:45.909459 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:06:45.909883 28551 external_mini_cluster.cc:1428] Setting key f92cc80e0131561697f25c5c9478b6b4
I20260806 21:06:45.913174 866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:45.917722 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:45.912615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46584 (local address 127.27.225.254:40121)
0806 21:06:45.913790 (+ 1175us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:45.913805 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:45.913914 (+ 109us) server_negotiation.cc:407] Connection header received
0806 21:06:45.914672 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:45.914696 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:45.915093 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:45.915466 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:45.916398 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:45.916404 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:45.916415 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:06:45.916427 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:45.916484 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:06:45.916506 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:45.916666 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:45.916740 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 21:06:45.916993 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":764,"thread_start_us":618,"threads_started":1}
I20260806 21:06:45.920701 866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f29d4d0e24804518a1d835a29200eb60: Bootstrap starting.
I20260806 21:06:45.923523 866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f29d4d0e24804518a1d835a29200eb60: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:45.924528 866 log.cc:824] T 00000000000000000000000000000000 P f29d4d0e24804518a1d835a29200eb60: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:45.926998 866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f29d4d0e24804518a1d835a29200eb60: No bootstrap required, opened a new log
I20260806 21:06:45.930673 866 raft_consensus.cc:348] T 00000000000000000000000000000000 P f29d4d0e24804518a1d835a29200eb60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f29d4d0e24804518a1d835a29200eb60" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40121 } }
I20260806 21:06:45.930922 866 raft_consensus.cc:374] T 00000000000000000000000000000000 P f29d4d0e24804518a1d835a29200eb60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:45.930968 866 raft_consensus.cc:729] T 00000000000000000000000000000000 P f29d4d0e24804518a1d835a29200eb60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f29d4d0e24804518a1d835a29200eb60, State: Initialized, Role: FOLLOWER
I20260806 21:06:45.931414 866 consensus_queue.cc:260] T 00000000000000000000000000000000 P f29d4d0e24804518a1d835a29200eb60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f29d4d0e24804518a1d835a29200eb60" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40121 } }
I20260806 21:06:45.931582 866 raft_consensus.cc:388] T 00000000000000000000000000000000 P f29d4d0e24804518a1d835a29200eb60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:06:45.931684 866 raft_consensus.cc:482] T 00000000000000000000000000000000 P f29d4d0e24804518a1d835a29200eb60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:06:45.931790 866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f29d4d0e24804518a1d835a29200eb60 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:45.932840 866 raft_consensus.cc:504] T 00000000000000000000000000000000 P f29d4d0e24804518a1d835a29200eb60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f29d4d0e24804518a1d835a29200eb60" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40121 } }
I20260806 21:06:45.933264 866 leader_election.cc:302] T 00000000000000000000000000000000 P f29d4d0e24804518a1d835a29200eb60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f29d4d0e24804518a1d835a29200eb60; no voters:
I20260806 21:06:45.933610 866 leader_election.cc:288] T 00000000000000000000000000000000 P f29d4d0e24804518a1d835a29200eb60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:06:45.933761 871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f29d4d0e24804518a1d835a29200eb60 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:45.934060 871 raft_consensus.cc:686] T 00000000000000000000000000000000 P f29d4d0e24804518a1d835a29200eb60 [term 1 LEADER]: Becoming Leader. State: Replica: f29d4d0e24804518a1d835a29200eb60, State: Running, Role: LEADER
I20260806 21:06:45.934432 871 consensus_queue.cc:237] T 00000000000000000000000000000000 P f29d4d0e24804518a1d835a29200eb60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f29d4d0e24804518a1d835a29200eb60" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40121 } }
I20260806 21:06:45.934962 866 sys_catalog.cc:564] T 00000000000000000000000000000000 P f29d4d0e24804518a1d835a29200eb60 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:06:45.936293 873 sys_catalog.cc:455] T 00000000000000000000000000000000 P f29d4d0e24804518a1d835a29200eb60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f29d4d0e24804518a1d835a29200eb60. Latest consensus state: current_term: 1 leader_uuid: "f29d4d0e24804518a1d835a29200eb60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f29d4d0e24804518a1d835a29200eb60" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40121 } } }
I20260806 21:06:45.936306 872 sys_catalog.cc:455] T 00000000000000000000000000000000 P f29d4d0e24804518a1d835a29200eb60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f29d4d0e24804518a1d835a29200eb60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f29d4d0e24804518a1d835a29200eb60" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40121 } } }
I20260806 21:06:45.936491 872 sys_catalog.cc:458] T 00000000000000000000000000000000 P f29d4d0e24804518a1d835a29200eb60 [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:45.936792 873 sys_catalog.cc:458] T 00000000000000000000000000000000 P f29d4d0e24804518a1d835a29200eb60 [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:45.937408 881 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:06:45.940194 881 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:06:45.946128 881 catalog_manager.cc:1347] Generated new cluster ID: 2681d187ed7049bebd836b91174e954a
I20260806 21:06:45.946229 881 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:06:45.965039 881 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:06:45.965472 881 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:06:45.970968 881 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f29d4d0e24804518a1d835a29200eb60: Generated new TSK 0
I20260806 21:06:45.971803 881 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:06:45.984174 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40121
--builtin_ntp_servers=127.27.225.212:37139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 21:06:46.112664 890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:46.112977 890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:46.113019 890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:46.117372 890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:46.117455 890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:46.117516 890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:06:46.122515 890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:37139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:46.123660 890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:46.125052 890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:46.135389 898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:46.135377 896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:46.135480 890 server_base.cc:1034] running on GCE node
W20260806 21:06:46.135371 895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:46.136255 890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:46.137008 890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:46.138187 890 hybrid_clock.cc:648] HybridClock initialized: now 1786050406138169 us; error 31 us; skew 500 ppm
I20260806 21:06:46.140405 890 webserver.cc:460] Webserver started at http://127.27.225.193:35043/ using document root <none> and password file <none>
I20260806 21:06:46.141055 890 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:46.141134 890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:46.141356 890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:46.143545 890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "302fdecd036242c6a609b629c428d37b"
format_stamp: "Formatted at 2026-08-06 21:06:46 on dist-test-slave-69rg"
server_key: "cef1e123b2679e40fd42149b8fbe75eb"
server_key_iv: "6d0670e67dca09a709d1f34e9001e0b5"
server_key_version: "encryptionkey@0"
I20260806 21:06:46.144203 890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "302fdecd036242c6a609b629c428d37b"
format_stamp: "Formatted at 2026-08-06 21:06:46 on dist-test-slave-69rg"
server_key: "cef1e123b2679e40fd42149b8fbe75eb"
server_key_iv: "6d0670e67dca09a709d1f34e9001e0b5"
server_key_version: "encryptionkey@0"
I20260806 21:06:46.148425 890 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:06:46.152004 904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:46.153851 890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 21:06:46.154045 890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "302fdecd036242c6a609b629c428d37b"
format_stamp: "Formatted at 2026-08-06 21:06:46 on dist-test-slave-69rg"
server_key: "cef1e123b2679e40fd42149b8fbe75eb"
server_key_iv: "6d0670e67dca09a709d1f34e9001e0b5"
server_key_version: "encryptionkey@0"
I20260806 21:06:46.154352 890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:46.160660 890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:46.161418 890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:46.161649 890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:46.162313 890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:46.163359 890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:46.163440 890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:46.163517 890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:46.163573 890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:46.174405 890 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:40523
I20260806 21:06:46.174425 1019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:40523 every 8 connection(s)
I20260806 21:06:46.175385 890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:06:46.181957 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:46.177308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:52453 (local address 127.27.225.254:40121)
0806 21:06:46.177465 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:46.177470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:46.178689 (+ 1219us) server_negotiation.cc:407] Connection header received
0806 21:06:46.180064 (+ 1375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:46.180067 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:46.180105 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:46.180192 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:46.181420 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:46.181422 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:46.181424 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:06:46.181435 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:46.181454 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:06:46.181462 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:46.181544 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:46.181728 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 21:06:46.181768 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 21:06:46.182330 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 890
I20260806 21:06:46.182451 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:06:46.182762 28551 external_mini_cluster.cc:1428] Setting key e4dbcb09984db46ad7683eb1a5945fc1
I20260806 21:06:46.183264 1022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:46.177724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40121 (local address 127.27.225.193:52453)
0806 21:06:46.178495 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:46.178540 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:46.179865 (+ 1325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:46.180371 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:46.180377 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:46.180478 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:46.181131 (+ 653us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:06:46.181133 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:46.181174 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:06:46.181243 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:46.181573 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:46.181582 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:46.181588 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 21:06:46.181757 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 21:06:46.182288 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":126,"threads_started":1}
I20260806 21:06:46.184993 1020 heartbeater.cc:347] Connected to a master server at 127.27.225.254:40121
I20260806 21:06:46.185328 1020 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:46.185870 1020 heartbeater.cc:511] Master 127.27.225.254:40121 requested a full tablet report, sending...
I20260806 21:06:46.186079 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40121
--builtin_ntp_servers=127.27.225.212:37139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 21:06:46.187778 823 ts_manager.cc:194] Registered new tserver with Master: 302fdecd036242c6a609b629c428d37b (127.27.225.193:40523)
W20260806 21:06:46.303926 1023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:46.304224 1023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:46.304273 1023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:46.308495 1023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:46.308583 1023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:46.308645 1023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:06:46.313709 1023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:37139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:46.314945 1023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:46.316540 1023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:46.326808 1028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:46.326898 1029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:46.326836 1031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:46.327100 1023 server_base.cc:1034] running on GCE node
I20260806 21:06:46.327881 1023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:46.328673 1023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:46.329941 1023 hybrid_clock.cc:648] HybridClock initialized: now 1786050406329922 us; error 54 us; skew 500 ppm
I20260806 21:06:46.332345 1023 webserver.cc:460] Webserver started at http://127.27.225.194:38143/ using document root <none> and password file <none>
I20260806 21:06:46.332978 1023 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:46.333078 1023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:46.333350 1023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:46.335662 1023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "fb3b57df5d604fdc8ea1e8d941c4fa89"
format_stamp: "Formatted at 2026-08-06 21:06:46 on dist-test-slave-69rg"
server_key: "925dc4b9c0cd2edcdd18aeccc776ffdc"
server_key_iv: "b25b276fe2bc9395b85ed4603608b8b9"
server_key_version: "encryptionkey@0"
I20260806 21:06:46.336308 1023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "fb3b57df5d604fdc8ea1e8d941c4fa89"
format_stamp: "Formatted at 2026-08-06 21:06:46 on dist-test-slave-69rg"
server_key: "925dc4b9c0cd2edcdd18aeccc776ffdc"
server_key_iv: "b25b276fe2bc9395b85ed4603608b8b9"
server_key_version: "encryptionkey@0"
I20260806 21:06:46.340427 1023 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:06:46.343446 1037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:46.344941 1023 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 21:06:46.345101 1023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "fb3b57df5d604fdc8ea1e8d941c4fa89"
format_stamp: "Formatted at 2026-08-06 21:06:46 on dist-test-slave-69rg"
server_key: "925dc4b9c0cd2edcdd18aeccc776ffdc"
server_key_iv: "b25b276fe2bc9395b85ed4603608b8b9"
server_key_version: "encryptionkey@0"
I20260806 21:06:46.345248 1023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:46.350288 1023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:46.350987 1023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:46.351202 1023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:46.351794 1023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:46.352778 1023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:46.352854 1023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:46.352931 1023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:46.352986 1023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:46.362844 1023 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:33471
I20260806 21:06:46.362866 1151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:33471 every 8 connection(s)
I20260806 21:06:46.363804 1023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:06:46.364353 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 1023
I20260806 21:06:46.364483 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:06:46.364835 28551 external_mini_cluster.cc:1428] Setting key b877ee93eae704f6f73284e6ed5cd5f6
I20260806 21:06:46.366967 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40121
--builtin_ntp_servers=127.27.225.212:37139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 21:06:46.372572 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:46.365728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:47523 (local address 127.27.225.254:40121)
0806 21:06:46.365895 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:46.365900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:46.369239 (+ 3339us) server_negotiation.cc:407] Connection header received
0806 21:06:46.370661 (+ 1422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:46.370664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:46.370708 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:46.370788 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:46.372040 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:46.372044 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:46.372046 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:06:46.372061 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:46.372089 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:06:46.372099 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:46.372200 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:46.372368 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 21:06:46.372420 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 21:06:46.373507 1154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:46.366220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40121 (local address 127.27.225.194:47523)
0806 21:06:46.369070 (+ 2850us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:46.369123 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:46.370436 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:46.371003 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:46.371010 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:46.371109 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:46.371761 (+ 652us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:06:46.371763 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:46.371807 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:06:46.371880 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:46.372199 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:46.372211 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:46.372220 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 21:06:46.372410 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 21:06:46.372677 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2459,"thread_start_us":120,"threads_started":1}
I20260806 21:06:46.374950 1152 heartbeater.cc:347] Connected to a master server at 127.27.225.254:40121
I20260806 21:06:46.375223 1152 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:46.375679 1152 heartbeater.cc:511] Master 127.27.225.254:40121 requested a full tablet report, sending...
I20260806 21:06:46.376998 823 ts_manager.cc:194] Registered new tserver with Master: fb3b57df5d604fdc8ea1e8d941c4fa89 (127.27.225.194:33471)
W20260806 21:06:46.482144 1155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:46.482429 1155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:46.482470 1155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:46.486562 1155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:46.486645 1155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:46.486706 1155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:06:46.491816 1155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:37139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:46.492921 1155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:46.494467 1155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:46.504331 1163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:46.504318 1160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:46.504395 1155 server_base.cc:1034] running on GCE node
W20260806 21:06:46.504292 1161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:46.505039 1155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:46.505724 1155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:46.506937 1155 hybrid_clock.cc:648] HybridClock initialized: now 1786050406506879 us; error 72 us; skew 500 ppm
I20260806 21:06:46.509169 1155 webserver.cc:460] Webserver started at http://127.27.225.195:45237/ using document root <none> and password file <none>
I20260806 21:06:46.509809 1155 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:46.509979 1155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:46.510206 1155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:46.512375 1155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "4a5cfa3752cc4a9baa3eb31d48db7d94"
format_stamp: "Formatted at 2026-08-06 21:06:46 on dist-test-slave-69rg"
server_key: "89a2e827ffce898a4ebe248ddc1705e1"
server_key_iv: "2ae37273d0de97cdf58e866f6de10e76"
server_key_version: "encryptionkey@0"
I20260806 21:06:46.512984 1155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "4a5cfa3752cc4a9baa3eb31d48db7d94"
format_stamp: "Formatted at 2026-08-06 21:06:46 on dist-test-slave-69rg"
server_key: "89a2e827ffce898a4ebe248ddc1705e1"
server_key_iv: "2ae37273d0de97cdf58e866f6de10e76"
server_key_version: "encryptionkey@0"
I20260806 21:06:46.517127 1155 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:06:46.520004 1169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:46.521255 1155 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 21:06:46.521405 1155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "4a5cfa3752cc4a9baa3eb31d48db7d94"
format_stamp: "Formatted at 2026-08-06 21:06:46 on dist-test-slave-69rg"
server_key: "89a2e827ffce898a4ebe248ddc1705e1"
server_key_iv: "2ae37273d0de97cdf58e866f6de10e76"
server_key_version: "encryptionkey@0"
I20260806 21:06:46.521534 1155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:46.526216 1155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:46.526877 1155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:46.527070 1155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:46.527689 1155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:46.528625 1155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:46.528695 1155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:46.528764 1155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:46.528817 1155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:46.538465 1155 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:37381
I20260806 21:06:46.538563 1285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:37381 every 8 connection(s)
I20260806 21:06:46.539494 1155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:06:46.544163 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 1155
I20260806 21:06:46.544301 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:06:46.544628 28551 external_mini_cluster.cc:1428] Setting key a388c20dd5e4a3a064940ea7f63d2fcb
I20260806 21:06:46.546797 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:46.541512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:33529 (local address 127.27.225.254:40121)
0806 21:06:46.541674 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:46.541679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:46.542900 (+ 1221us) server_negotiation.cc:407] Connection header received
0806 21:06:46.544611 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:46.544615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:46.544660 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:46.544777 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:46.546302 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:46.546305 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:46.546307 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:06:46.546318 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:46.546338 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:06:46.546346 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:46.546440 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:46.546611 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 21:06:46.546658 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 21:06:46.547744 1289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:46.542012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40121 (local address 127.27.225.195:33529)
0806 21:06:46.542736 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:46.542779 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:46.544310 (+ 1531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:46.545073 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:46.545082 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:46.545215 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:46.545950 (+ 735us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:06:46.545953 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:46.546010 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:06:46.546108 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:46.546440 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:46.546452 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:46.546461 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 21:06:46.546663 (+ 202us) client_negotiation.cc:240] Negotiation successful
0806 21:06:46.546957 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":141,"threads_started":1}
I20260806 21:06:46.549032 1286 heartbeater.cc:347] Connected to a master server at 127.27.225.254:40121
I20260806 21:06:46.549348 1286 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:46.549878 1286 heartbeater.cc:511] Master 127.27.225.254:40121 requested a full tablet report, sending...
I20260806 21:06:46.551052 823 ts_manager.cc:194] Registered new tserver with Master: 4a5cfa3752cc4a9baa3eb31d48db7d94 (127.27.225.195:37381)
I20260806 21:06:46.559041 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:06:46.562700 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:46.561176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46588 (local address 127.27.225.254:40121)
0806 21:06:46.561306 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:46.561310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:46.561474 (+ 164us) server_negotiation.cc:407] Connection header received
0806 21:06:46.561564 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:46.561566 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:46.561600 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:46.561668 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:46.562381 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:46.562384 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:46.562385 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:06:46.562396 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:46.562415 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:06:46.562423 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:46.562489 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:46.562558 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 21:06:46.562591 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 21:06:46.565654 823 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:06:46.568816 823 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:06:46.581022 1304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:46.578737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40842 (local address 127.27.225.194:33471)
0806 21:06:46.579166 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:46.579185 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:46.579208 (+ 23us) server_negotiation.cc:407] Connection header received
0806 21:06:46.579447 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:46.579454 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:46.579649 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:46.579795 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:46.580409 (+ 614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:46.580415 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:46.580427 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:06:46.580437 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:46.580479 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:06:46.580507 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:46.580628 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:46.580787 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 21:06:46.580862 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":118,"threads_started":1}
I20260806 21:06:46.580996 1300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:46.577575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:37381 (local address 127.0.0.1:54106)
0806 21:06:46.578256 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:46.578307 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:46.578407 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:46.579373 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:46.579376 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:46.579393 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:46.580016 (+ 623us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:06:46.580018 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:46.580039 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:06:46.580113 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:46.580624 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:46.580627 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:46.580631 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 21:06:46.580744 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 21:06:46.580814 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":88,"threads_started":1}
I20260806 21:06:46.581167 1299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:46.577684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:40523 (local address 127.0.0.1:57306)
0806 21:06:46.578374 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:46.578391 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:46.578480 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:46.579278 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:46.579285 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:46.579367 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:46.580301 (+ 934us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:06:46.580303 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:46.580314 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:06:46.580343 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:46.580951 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:46.580954 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:46.580957 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 21:06:46.581009 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 21:06:46.581044 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":147,"threads_started":1}
I20260806 21:06:46.581427 1303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:46.578345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57306 (local address 127.27.225.193:40523)
0806 21:06:46.578793 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:46.578805 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:46.578830 (+ 25us) server_negotiation.cc:407] Connection header received
0806 21:06:46.578888 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:46.578895 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:46.579094 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:46.580137 (+ 1043us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:46.580748 (+ 611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:46.580753 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:46.580763 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:06:46.580774 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:46.580811 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:06:46.580839 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:46.581134 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:46.581181 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 21:06:46.581256 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":114,"threads_started":1}
I20260806 21:06:46.581022 1302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:46.578329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:33471 (local address 127.0.0.1:40842)
0806 21:06:46.578797 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:46.578812 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:46.579307 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:46.579778 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:46.579780 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:46.579794 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:46.580273 (+ 479us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:06:46.580275 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:46.580285 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:06:46.580306 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:46.580612 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:46.580622 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:46.580629 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 21:06:46.580756 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 21:06:46.580817 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":87,"threads_started":1}
I20260806 21:06:46.584146 953 tablet_service.cc:1467] Processing CreateTablet for tablet 7d4cc17be7474d20a1774d30c8974995 (DEFAULT_TABLE table=test-table [id=41dca6a8dea0430aac6e3e77513f5595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:46.584146 1086 tablet_service.cc:1467] Processing CreateTablet for tablet 7d4cc17be7474d20a1774d30c8974995 (DEFAULT_TABLE table=test-table [id=41dca6a8dea0430aac6e3e77513f5595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:46.582368 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:46.577719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54106 (local address 127.27.225.195:37381)
0806 21:06:46.579025 (+ 1306us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:46.579035 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:46.579055 (+ 20us) server_negotiation.cc:407] Connection header received
0806 21:06:46.579103 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:46.579108 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:46.579263 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:46.579391 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:46.580425 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:46.580430 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:46.580439 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:06:46.580451 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:46.580486 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:06:46.580517 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:46.582107 (+ 1590us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:46.582155 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 21:06:46.582218 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1213,"thread_start_us":174,"threads_started":1}
I20260806 21:06:46.585222 1086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d4cc17be7474d20a1774d30c8974995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:46.585227 953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d4cc17be7474d20a1774d30c8974995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:46.587519 1219 tablet_service.cc:1467] Processing CreateTablet for tablet 7d4cc17be7474d20a1774d30c8974995 (DEFAULT_TABLE table=test-table [id=41dca6a8dea0430aac6e3e77513f5595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:46.588578 1219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d4cc17be7474d20a1774d30c8974995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:46.591463 1305 tablet_bootstrap.cc:492] T 7d4cc17be7474d20a1774d30c8974995 P fb3b57df5d604fdc8ea1e8d941c4fa89: Bootstrap starting.
I20260806 21:06:46.593727 1305 tablet_bootstrap.cc:654] T 7d4cc17be7474d20a1774d30c8974995 P fb3b57df5d604fdc8ea1e8d941c4fa89: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:46.594002 1306 tablet_bootstrap.cc:492] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b: Bootstrap starting.
I20260806 21:06:46.594719 1305 log.cc:824] T 7d4cc17be7474d20a1774d30c8974995 P fb3b57df5d604fdc8ea1e8d941c4fa89: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:46.596172 1308 tablet_bootstrap.cc:492] T 7d4cc17be7474d20a1774d30c8974995 P 4a5cfa3752cc4a9baa3eb31d48db7d94: Bootstrap starting.
I20260806 21:06:46.596333 1306 tablet_bootstrap.cc:654] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:46.596968 1305 tablet_bootstrap.cc:492] T 7d4cc17be7474d20a1774d30c8974995 P fb3b57df5d604fdc8ea1e8d941c4fa89: No bootstrap required, opened a new log
I20260806 21:06:46.597182 1305 ts_tablet_manager.cc:1397] T 7d4cc17be7474d20a1774d30c8974995 P fb3b57df5d604fdc8ea1e8d941c4fa89: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:06:46.597195 1306 log.cc:824] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:46.598228 1308 tablet_bootstrap.cc:654] T 7d4cc17be7474d20a1774d30c8974995 P 4a5cfa3752cc4a9baa3eb31d48db7d94: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:46.598901 1308 log.cc:824] T 7d4cc17be7474d20a1774d30c8974995 P 4a5cfa3752cc4a9baa3eb31d48db7d94: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:46.599272 1306 tablet_bootstrap.cc:492] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b: No bootstrap required, opened a new log
I20260806 21:06:46.599546 1306 ts_tablet_manager.cc:1397] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 21:06:46.600814 1308 tablet_bootstrap.cc:492] T 7d4cc17be7474d20a1774d30c8974995 P 4a5cfa3752cc4a9baa3eb31d48db7d94: No bootstrap required, opened a new log
I20260806 21:06:46.601012 1308 ts_tablet_manager.cc:1397] T 7d4cc17be7474d20a1774d30c8974995 P 4a5cfa3752cc4a9baa3eb31d48db7d94: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260806 21:06:46.601929 1305 raft_consensus.cc:348] T 7d4cc17be7474d20a1774d30c8974995 P fb3b57df5d604fdc8ea1e8d941c4fa89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5cfa3752cc4a9baa3eb31d48db7d94" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 37381 } } peers { permanent_uuid: "302fdecd036242c6a609b629c428d37b" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40523 } } peers { permanent_uuid: "fb3b57df5d604fdc8ea1e8d941c4fa89" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33471 } }
I20260806 21:06:46.602211 1305 raft_consensus.cc:374] T 7d4cc17be7474d20a1774d30c8974995 P fb3b57df5d604fdc8ea1e8d941c4fa89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:46.602301 1305 raft_consensus.cc:729] T 7d4cc17be7474d20a1774d30c8974995 P fb3b57df5d604fdc8ea1e8d941c4fa89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb3b57df5d604fdc8ea1e8d941c4fa89, State: Initialized, Role: FOLLOWER
I20260806 21:06:46.602830 1305 consensus_queue.cc:260] T 7d4cc17be7474d20a1774d30c8974995 P fb3b57df5d604fdc8ea1e8d941c4fa89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5cfa3752cc4a9baa3eb31d48db7d94" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 37381 } } peers { permanent_uuid: "302fdecd036242c6a609b629c428d37b" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40523 } } peers { permanent_uuid: "fb3b57df5d604fdc8ea1e8d941c4fa89" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33471 } }
I20260806 21:06:46.603626 1152 heartbeater.cc:503] Master 127.27.225.254:40121 was elected leader, sending a full tablet report...
I20260806 21:06:46.603946 1305 ts_tablet_manager.cc:1428] T 7d4cc17be7474d20a1774d30c8974995 P fb3b57df5d604fdc8ea1e8d941c4fa89: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 21:06:46.604959 1308 raft_consensus.cc:348] T 7d4cc17be7474d20a1774d30c8974995 P 4a5cfa3752cc4a9baa3eb31d48db7d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5cfa3752cc4a9baa3eb31d48db7d94" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 37381 } } peers { permanent_uuid: "302fdecd036242c6a609b629c428d37b" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40523 } } peers { permanent_uuid: "fb3b57df5d604fdc8ea1e8d941c4fa89" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33471 } }
I20260806 21:06:46.605242 1308 raft_consensus.cc:374] T 7d4cc17be7474d20a1774d30c8974995 P 4a5cfa3752cc4a9baa3eb31d48db7d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:46.604990 1306 raft_consensus.cc:348] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5cfa3752cc4a9baa3eb31d48db7d94" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 37381 } } peers { permanent_uuid: "302fdecd036242c6a609b629c428d37b" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40523 } } peers { permanent_uuid: "fb3b57df5d604fdc8ea1e8d941c4fa89" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33471 } }
I20260806 21:06:46.605325 1306 raft_consensus.cc:374] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:46.605341 1308 raft_consensus.cc:729] T 7d4cc17be7474d20a1774d30c8974995 P 4a5cfa3752cc4a9baa3eb31d48db7d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a5cfa3752cc4a9baa3eb31d48db7d94, State: Initialized, Role: FOLLOWER
I20260806 21:06:46.605417 1306 raft_consensus.cc:729] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 302fdecd036242c6a609b629c428d37b, State: Initialized, Role: FOLLOWER
I20260806 21:06:46.605804 1308 consensus_queue.cc:260] T 7d4cc17be7474d20a1774d30c8974995 P 4a5cfa3752cc4a9baa3eb31d48db7d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5cfa3752cc4a9baa3eb31d48db7d94" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 37381 } } peers { permanent_uuid: "302fdecd036242c6a609b629c428d37b" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40523 } } peers { permanent_uuid: "fb3b57df5d604fdc8ea1e8d941c4fa89" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33471 } }
I20260806 21:06:46.605909 1306 consensus_queue.cc:260] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5cfa3752cc4a9baa3eb31d48db7d94" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 37381 } } peers { permanent_uuid: "302fdecd036242c6a609b629c428d37b" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40523 } } peers { permanent_uuid: "fb3b57df5d604fdc8ea1e8d941c4fa89" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33471 } }
I20260806 21:06:46.606514 1286 heartbeater.cc:503] Master 127.27.225.254:40121 was elected leader, sending a full tablet report...
I20260806 21:06:46.606889 1306 ts_tablet_manager.cc:1428] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 21:06:46.606894 1308 ts_tablet_manager.cc:1428] T 7d4cc17be7474d20a1774d30c8974995 P 4a5cfa3752cc4a9baa3eb31d48db7d94: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260806 21:06:46.607299 1020 heartbeater.cc:503] Master 127.27.225.254:40121 was elected leader, sending a full tablet report...
W20260806 21:06:46.614845 1153 tablet.cc:2369] T 7d4cc17be7474d20a1774d30c8974995 P fb3b57df5d604fdc8ea1e8d941c4fa89: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 21:06:46.676606 1021 tablet.cc:2369] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:06:46.716466 1313 raft_consensus.cc:482] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:06:46.716809 1313 raft_consensus.cc:504] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5cfa3752cc4a9baa3eb31d48db7d94" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 37381 } } peers { permanent_uuid: "302fdecd036242c6a609b629c428d37b" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40523 } } peers { permanent_uuid: "fb3b57df5d604fdc8ea1e8d941c4fa89" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33471 } }
I20260806 21:06:46.718051 1313 leader_election.cc:288] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a5cfa3752cc4a9baa3eb31d48db7d94 (127.27.225.195:37381), fb3b57df5d604fdc8ea1e8d941c4fa89 (127.27.225.194:33471)
I20260806 21:06:46.720397 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:46.718509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:59259 (local address 127.27.225.195:37381)
0806 21:06:46.718768 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:46.718775 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:46.718792 (+ 17us) server_negotiation.cc:407] Connection header received
0806 21:06:46.718959 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:46.718962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:46.719005 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:46.719097 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:46.719995 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:46.719998 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:46.720001 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:06:46.720013 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:46.720036 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:06:46.720051 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:46.720157 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:46.720216 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 21:06:46.720259 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
I20260806 21:06:46.720458 1314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:46.718381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:37381 (local address 127.27.225.193:59259)
0806 21:06:46.718724 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:46.718738 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:46.718833 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:46.719117 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:46.719120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:46.719134 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:46.719819 (+ 685us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:06:46.719821 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:46.719835 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:06:46.719865 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:46.720142 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:46.720144 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:46.720148 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 21:06:46.720224 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 21:06:46.720259 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":37,"thread_start_us":103,"threads_started":1}
I20260806 21:06:46.720464 1304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:46.718496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:54347 (local address 127.27.225.194:33471)
0806 21:06:46.718644 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:46.718649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:46.718821 (+ 172us) server_negotiation.cc:407] Connection header received
0806 21:06:46.718959 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:46.718962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:46.719008 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:46.719100 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:46.720016 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:46.720020 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:46.720023 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:06:46.720043 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:46.720064 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:06:46.720073 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:46.720186 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:46.720272 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 21:06:46.720315 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 21:06:46.720506 1315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:46.718381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:33471 (local address 127.27.225.193:54347)
0806 21:06:46.718696 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:46.718714 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:46.718848 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:46.719120 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:46.719123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:46.719138 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:46.719759 (+ 621us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:06:46.719788 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:46.719800 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:06:46.719829 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:46.720177 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:46.720181 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:46.720184 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 21:06:46.720273 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 21:06:46.720314 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":88,"threads_started":1}
I20260806 21:06:46.721324 1239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d4cc17be7474d20a1774d30c8974995" candidate_uuid: "302fdecd036242c6a609b629c428d37b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a5cfa3752cc4a9baa3eb31d48db7d94" is_pre_election: true
I20260806 21:06:46.721547 1106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d4cc17be7474d20a1774d30c8974995" candidate_uuid: "302fdecd036242c6a609b629c428d37b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb3b57df5d604fdc8ea1e8d941c4fa89" is_pre_election: true
I20260806 21:06:46.721678 1239 raft_consensus.cc:2454] T 7d4cc17be7474d20a1774d30c8974995 P 4a5cfa3752cc4a9baa3eb31d48db7d94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 302fdecd036242c6a609b629c428d37b in term 0.
I20260806 21:06:46.721882 1106 raft_consensus.cc:2454] T 7d4cc17be7474d20a1774d30c8974995 P fb3b57df5d604fdc8ea1e8d941c4fa89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 302fdecd036242c6a609b629c428d37b in term 0.
I20260806 21:06:46.722294 905 leader_election.cc:302] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 302fdecd036242c6a609b629c428d37b, 4a5cfa3752cc4a9baa3eb31d48db7d94, fb3b57df5d604fdc8ea1e8d941c4fa89; no voters:
I20260806 21:06:46.722601 1313 raft_consensus.cc:2781] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:06:46.722712 1313 raft_consensus.cc:482] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:06:46.722749 1313 raft_consensus.cc:3037] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:46.723923 1313 raft_consensus.cc:504] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5cfa3752cc4a9baa3eb31d48db7d94" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 37381 } } peers { permanent_uuid: "302fdecd036242c6a609b629c428d37b" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40523 } } peers { permanent_uuid: "fb3b57df5d604fdc8ea1e8d941c4fa89" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33471 } }
I20260806 21:06:46.724294 1313 leader_election.cc:288] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b [CANDIDATE]: Term 1 election: Requested vote from peers 4a5cfa3752cc4a9baa3eb31d48db7d94 (127.27.225.195:37381), fb3b57df5d604fdc8ea1e8d941c4fa89 (127.27.225.194:33471)
I20260806 21:06:46.724686 1106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d4cc17be7474d20a1774d30c8974995" candidate_uuid: "302fdecd036242c6a609b629c428d37b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb3b57df5d604fdc8ea1e8d941c4fa89"
I20260806 21:06:46.724709 1239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d4cc17be7474d20a1774d30c8974995" candidate_uuid: "302fdecd036242c6a609b629c428d37b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a5cfa3752cc4a9baa3eb31d48db7d94"
I20260806 21:06:46.724841 1239 raft_consensus.cc:3037] T 7d4cc17be7474d20a1774d30c8974995 P 4a5cfa3752cc4a9baa3eb31d48db7d94 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:46.724831 1106 raft_consensus.cc:3037] T 7d4cc17be7474d20a1774d30c8974995 P fb3b57df5d604fdc8ea1e8d941c4fa89 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:46.726187 1239 raft_consensus.cc:2454] T 7d4cc17be7474d20a1774d30c8974995 P 4a5cfa3752cc4a9baa3eb31d48db7d94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 302fdecd036242c6a609b629c428d37b in term 1.
I20260806 21:06:46.726289 1106 raft_consensus.cc:2454] T 7d4cc17be7474d20a1774d30c8974995 P fb3b57df5d604fdc8ea1e8d941c4fa89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 302fdecd036242c6a609b629c428d37b in term 1.
I20260806 21:06:46.726609 905 leader_election.cc:302] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 302fdecd036242c6a609b629c428d37b, 4a5cfa3752cc4a9baa3eb31d48db7d94; no voters:
I20260806 21:06:46.726810 1313 raft_consensus.cc:2781] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:46.727094 1313 raft_consensus.cc:686] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b [term 1 LEADER]: Becoming Leader. State: Replica: 302fdecd036242c6a609b629c428d37b, State: Running, Role: LEADER
I20260806 21:06:46.727459 1313 consensus_queue.cc:237] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5cfa3752cc4a9baa3eb31d48db7d94" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 37381 } } peers { permanent_uuid: "302fdecd036242c6a609b629c428d37b" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40523 } } peers { permanent_uuid: "fb3b57df5d604fdc8ea1e8d941c4fa89" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33471 } }
I20260806 21:06:46.731091 826 catalog_manager.cc:5495] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b reported cstate change: term changed from 0 to 1, leader changed from <none> to 302fdecd036242c6a609b629c428d37b (127.27.225.193). New cstate: current_term: 1 leader_uuid: "302fdecd036242c6a609b629c428d37b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a5cfa3752cc4a9baa3eb31d48db7d94" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 37381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "302fdecd036242c6a609b629c428d37b" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb3b57df5d604fdc8ea1e8d941c4fa89" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33471 } health_report { overall_health: UNKNOWN } } }
I20260806 21:06:46.772979 1303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:46.771353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57308 (local address 127.27.225.193:40523)
0806 21:06:46.771511 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:46.771515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:46.771529 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:06:46.771615 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:46.771617 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:46.771678 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:46.771756 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:46.772569 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:46.772596 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:46.772598 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:06:46.772610 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:46.772629 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:06:46.772638 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:46.772717 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:46.772835 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 21:06:46.772871 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 21:06:46.779208 1106 raft_consensus.cc:1261] T 7d4cc17be7474d20a1774d30c8974995 P fb3b57df5d604fdc8ea1e8d941c4fa89 [term 1 FOLLOWER]: Refusing update from remote peer 302fdecd036242c6a609b629c428d37b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:06:46.779222 1239 raft_consensus.cc:1261] T 7d4cc17be7474d20a1774d30c8974995 P 4a5cfa3752cc4a9baa3eb31d48db7d94 [term 1 FOLLOWER]: Refusing update from remote peer 302fdecd036242c6a609b629c428d37b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:06:46.780018 1313 consensus_queue.cc:1035] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb3b57df5d604fdc8ea1e8d941c4fa89" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:06:46.780282 1316 consensus_queue.cc:1035] T 7d4cc17be7474d20a1774d30c8974995 P 302fdecd036242c6a609b629c428d37b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a5cfa3752cc4a9baa3eb31d48db7d94" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 37381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:06:46.788056 1319 mvcc.cc:205] Tried to move back new op lower bound from 7315662466159697920 to 7315662465958989824. Current Snapshot: MvccSnapshot[applied={T|T < 7315662466159697920}]
I20260806 21:06:46.789014 1321 mvcc.cc:205] Tried to move back new op lower bound from 7315662466159697920 to 7315662465958989824. Current Snapshot: MvccSnapshot[applied={T|T < 7315662466159697920}]
I20260806 21:06:46.790657 1320 mvcc.cc:205] Tried to move back new op lower bound from 7315662466159697920 to 7315662465958989824. Current Snapshot: MvccSnapshot[applied={T|T < 7315662466159697920}]
I20260806 21:06:46.798803 826 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:06:46.799010 826 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:06:46.801753 826 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f29d4d0e24804518a1d835a29200eb60: Sending DeleteTablet for 3 replicas of tablet 7d4cc17be7474d20a1774d30c8974995
I20260806 21:06:46.802724 953 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d4cc17be7474d20a1774d30c8974995 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:46 UTC) from {username='slave'} at 127.0.0.1:57306
I20260806 21:06:46.802867 1219 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d4cc17be7474d20a1774d30c8974995 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:46 UTC) from {username='slave'} at 127.0.0.1:54106
I20260806 21:06:46.803313 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 890
I20260806 21:06:46.803407 1086 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d4cc17be7474d20a1774d30c8974995 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:46 UTC) from {username='slave'} at 127.0.0.1:40842
I20260806 21:06:46.803978 1330 tablet_replica.cc:330] T 7d4cc17be7474d20a1774d30c8974995 P 4a5cfa3752cc4a9baa3eb31d48db7d94: stopping tablet replica
I20260806 21:06:46.804323 1330 raft_consensus.cc:2229] T 7d4cc17be7474d20a1774d30c8974995 P 4a5cfa3752cc4a9baa3eb31d48db7d94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:06:46.804649 1330 raft_consensus.cc:2258] T 7d4cc17be7474d20a1774d30c8974995 P 4a5cfa3752cc4a9baa3eb31d48db7d94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:46.806159 1330 ts_tablet_manager.cc:1905] T 7d4cc17be7474d20a1774d30c8974995 P 4a5cfa3752cc4a9baa3eb31d48db7d94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:46.806176 1331 tablet_replica.cc:330] T 7d4cc17be7474d20a1774d30c8974995 P fb3b57df5d604fdc8ea1e8d941c4fa89: stopping tablet replica
I20260806 21:06:46.806495 1331 raft_consensus.cc:2229] T 7d4cc17be7474d20a1774d30c8974995 P fb3b57df5d604fdc8ea1e8d941c4fa89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:06:46.806793 1331 raft_consensus.cc:2258] T 7d4cc17be7474d20a1774d30c8974995 P fb3b57df5d604fdc8ea1e8d941c4fa89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:46.808028 1331 ts_tablet_manager.cc:1905] T 7d4cc17be7474d20a1774d30c8974995 P fb3b57df5d604fdc8ea1e8d941c4fa89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:46.809607 1330 ts_tablet_manager.cc:1918] T 7d4cc17be7474d20a1774d30c8974995 P 4a5cfa3752cc4a9baa3eb31d48db7d94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:06:46.809718 1330 log.cc:1196] T 7d4cc17be7474d20a1774d30c8974995 P 4a5cfa3752cc4a9baa3eb31d48db7d94: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/7d4cc17be7474d20a1774d30c8974995
I20260806 21:06:46.810137 1330 ts_tablet_manager.cc:1939] T 7d4cc17be7474d20a1774d30c8974995 P 4a5cfa3752cc4a9baa3eb31d48db7d94: Deleting consensus metadata
I20260806 21:06:46.810726 1331 ts_tablet_manager.cc:1918] T 7d4cc17be7474d20a1774d30c8974995 P fb3b57df5d604fdc8ea1e8d941c4fa89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:06:46.810858 1331 log.cc:1196] T 7d4cc17be7474d20a1774d30c8974995 P fb3b57df5d604fdc8ea1e8d941c4fa89: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050378084738-28551-0/minicluster-data/ts-1/wal/wals/7d4cc17be7474d20a1774d30c8974995
I20260806 21:06:46.811226 1331 ts_tablet_manager.cc:1939] T 7d4cc17be7474d20a1774d30c8974995 P fb3b57df5d604fdc8ea1e8d941c4fa89: Deleting consensus metadata
I20260806 21:06:46.811575 813 catalog_manager.cc:4841] TS 4a5cfa3752cc4a9baa3eb31d48db7d94 (127.27.225.195:37381): tablet 7d4cc17be7474d20a1774d30c8974995 (table test-table [id=41dca6a8dea0430aac6e3e77513f5595]) successfully deleted
W20260806 21:06:46.812228 816 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.225.193:40523 (error 108)
I20260806 21:06:46.812716 816 catalog_manager.cc:4841] TS fb3b57df5d604fdc8ea1e8d941c4fa89 (127.27.225.194:33471): tablet 7d4cc17be7474d20a1774d30c8974995 (table test-table [id=41dca6a8dea0430aac6e3e77513f5595]) successfully deleted
W20260806 21:06:46.813383 1302 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:06:46.813023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:40523 (local address 127.0.0.1:57316)
0806 21:06:46.813206 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:46.813275 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.193:40523: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 21:06:46.813520 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 1023
W20260806 21:06:46.813571 816 catalog_manager.cc:4584] TS 302fdecd036242c6a609b629c428d37b (127.27.225.193:40523): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7d4cc17be7474d20a1774d30c8974995: Network error: Client connection negotiation failed: client connection to 127.27.225.193:40523: connect: Connection reset by peer (error 104)
I20260806 21:06:46.823177 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 1155
I20260806 21:06:46.829703 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 798
2026-08-06T21:06:46Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1141 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 21:06:46.853291 28551 mini_oidc.cc:152] Starting JWKS server
I20260806 21:06:46.854997 28551 webserver.cc:460] Webserver started at https://127.0.0.1:32815/ using document root <none> and password file <none>
I20260806 21:06:46.855170 28551 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 21:06:46.855458 28551 webserver.cc:460] Webserver started at http://127.0.0.1:46329/ using document root <none> and password file <none>
2026-08-06T21:06:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:06:46Z Disabled control of system clock
I20260806 21:06:46.866005 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:41039
--webserver_interface=127.27.225.254
--webserver_port=0
--builtin_ntp_servers=127.27.225.212:40925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:41039
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32815/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050378084738-28551-0/testchainca.cert with env {}
W20260806 21:06:46.979877 1338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:46.980167 1338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:46.980211 1338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:46.984369 1338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:06:46.984442 1338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:46.984458 1338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:46.984476 1338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:06:46.984495 1338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 21:06:46.984512 1338 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 21:06:46.984526 1338 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:32815/jwks.json
I20260806 21:06:46.990145 1338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:40925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:41039
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:41039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32815/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050378084738-28551-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:46.991220 1338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:46.992209 1338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:47.001284 1344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:47.001322 1346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:47.001288 1343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:47.001866 1338 server_base.cc:1034] running on GCE node
I20260806 21:06:47.002440 1338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:47.003582 1338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:47.004807 1338 hybrid_clock.cc:648] HybridClock initialized: now 1786050407004760 us; error 49 us; skew 500 ppm
I20260806 21:06:47.006979 1338 webserver.cc:460] Webserver started at http://127.27.225.254:43441/ using document root <none> and password file <none>
I20260806 21:06:47.007651 1338 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:47.007764 1338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:47.008023 1338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:47.010416 1338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "cecb37cf9492411bbbc66790899b0c7c"
format_stamp: "Formatted at 2026-08-06 21:06:47 on dist-test-slave-69rg"
server_key: "2b1e3ce463d323074071082609a81580"
server_key_iv: "9f1aea1a288c155c27dc986008a56296"
server_key_version: "encryptionkey@0"
I20260806 21:06:47.011078 1338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "cecb37cf9492411bbbc66790899b0c7c"
format_stamp: "Formatted at 2026-08-06 21:06:47 on dist-test-slave-69rg"
server_key: "2b1e3ce463d323074071082609a81580"
server_key_iv: "9f1aea1a288c155c27dc986008a56296"
server_key_version: "encryptionkey@0"
I20260806 21:06:47.015123 1338 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:06:47.018349 1353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:47.019881 1338 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 21:06:47.020018 1338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "cecb37cf9492411bbbc66790899b0c7c"
format_stamp: "Formatted at 2026-08-06 21:06:47 on dist-test-slave-69rg"
server_key: "2b1e3ce463d323074071082609a81580"
server_key_iv: "9f1aea1a288c155c27dc986008a56296"
server_key_version: "encryptionkey@0"
I20260806 21:06:47.020135 1338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:47.035267 1338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:47.036010 1338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:47.036187 1338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:47.044302 1338 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:41039
I20260806 21:06:47.044371 1406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:41039 every 8 connection(s)
I20260806 21:06:47.045401 1338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:06:47.048885 1407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:47.052783 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 1338
I20260806 21:06:47.052923 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:06:47.053229 28551 external_mini_cluster.cc:1428] Setting key 013416ce49f9092d6a5b220c23823faa
I20260806 21:06:47.055500 1407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cecb37cf9492411bbbc66790899b0c7c: Bootstrap starting.
I20260806 21:06:47.058498 1407 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cecb37cf9492411bbbc66790899b0c7c: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:47.059399 1407 log.cc:824] T 00000000000000000000000000000000 P cecb37cf9492411bbbc66790899b0c7c: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:47.061707 1407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cecb37cf9492411bbbc66790899b0c7c: No bootstrap required, opened a new log
I20260806 21:06:47.064052 1410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:47.054768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35240 (local address 127.27.225.254:41039)
0806 21:06:47.055533 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:47.055556 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:47.055604 (+ 48us) server_negotiation.cc:407] Connection header received
0806 21:06:47.056492 (+ 888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:47.056518 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:47.057074 (+ 556us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:47.057469 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:47.057976 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:47.059477 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:47.060407 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:47.061072 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:47.061254 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:47.062196 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:47.062203 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:47.062216 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:06:47.062610 (+ 394us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:47.062669 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:06:47.062688 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:47.062793 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:47.062964 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 21:06:47.063278 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":147,"threads_started":1}
I20260806 21:06:47.065999 1407 raft_consensus.cc:348] T 00000000000000000000000000000000 P cecb37cf9492411bbbc66790899b0c7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cecb37cf9492411bbbc66790899b0c7c" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41039 } }
I20260806 21:06:47.066299 1407 raft_consensus.cc:374] T 00000000000000000000000000000000 P cecb37cf9492411bbbc66790899b0c7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:47.066401 1407 raft_consensus.cc:729] T 00000000000000000000000000000000 P cecb37cf9492411bbbc66790899b0c7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cecb37cf9492411bbbc66790899b0c7c, State: Initialized, Role: FOLLOWER
I20260806 21:06:47.066916 1407 consensus_queue.cc:260] T 00000000000000000000000000000000 P cecb37cf9492411bbbc66790899b0c7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cecb37cf9492411bbbc66790899b0c7c" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41039 } }
I20260806 21:06:47.067045 1407 raft_consensus.cc:388] T 00000000000000000000000000000000 P cecb37cf9492411bbbc66790899b0c7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:06:47.067128 1407 raft_consensus.cc:482] T 00000000000000000000000000000000 P cecb37cf9492411bbbc66790899b0c7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:06:47.067252 1407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cecb37cf9492411bbbc66790899b0c7c [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:47.067425 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:06:47.068619 1407 raft_consensus.cc:504] T 00000000000000000000000000000000 P cecb37cf9492411bbbc66790899b0c7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cecb37cf9492411bbbc66790899b0c7c" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41039 } }
I20260806 21:06:47.068985 1407 leader_election.cc:302] T 00000000000000000000000000000000 P cecb37cf9492411bbbc66790899b0c7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cecb37cf9492411bbbc66790899b0c7c; no voters:
I20260806 21:06:47.069370 1407 leader_election.cc:288] T 00000000000000000000000000000000 P cecb37cf9492411bbbc66790899b0c7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:06:47.069509 1412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cecb37cf9492411bbbc66790899b0c7c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:47.069772 1412 raft_consensus.cc:686] T 00000000000000000000000000000000 P cecb37cf9492411bbbc66790899b0c7c [term 1 LEADER]: Becoming Leader. State: Replica: cecb37cf9492411bbbc66790899b0c7c, State: Running, Role: LEADER
I20260806 21:06:47.070152 1412 consensus_queue.cc:237] T 00000000000000000000000000000000 P cecb37cf9492411bbbc66790899b0c7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cecb37cf9492411bbbc66790899b0c7c" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41039 } }
I20260806 21:06:47.070689 1407 sys_catalog.cc:564] T 00000000000000000000000000000000 P cecb37cf9492411bbbc66790899b0c7c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:06:47.071893 1414 sys_catalog.cc:455] T 00000000000000000000000000000000 P cecb37cf9492411bbbc66790899b0c7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader cecb37cf9492411bbbc66790899b0c7c. Latest consensus state: current_term: 1 leader_uuid: "cecb37cf9492411bbbc66790899b0c7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cecb37cf9492411bbbc66790899b0c7c" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41039 } } }
I20260806 21:06:47.072094 1414 sys_catalog.cc:458] T 00000000000000000000000000000000 P cecb37cf9492411bbbc66790899b0c7c [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:47.071906 1413 sys_catalog.cc:455] T 00000000000000000000000000000000 P cecb37cf9492411bbbc66790899b0c7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cecb37cf9492411bbbc66790899b0c7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cecb37cf9492411bbbc66790899b0c7c" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41039 } } }
I20260806 21:06:47.072229 1413 sys_catalog.cc:458] T 00000000000000000000000000000000 P cecb37cf9492411bbbc66790899b0c7c [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:47.072633 1422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:06:47.076054 1422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:06:47.081921 1422 catalog_manager.cc:1347] Generated new cluster ID: f7564111dc4242d290aac2b8175de4aa
I20260806 21:06:47.082013 1422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:06:47.100742 1422 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:06:47.101713 1422 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:06:47.107278 1422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cecb37cf9492411bbbc66790899b0c7c: Generated new TSK 0
I20260806 21:06:47.108071 1422 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 21:06:47.147686 1438 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 21:06:47.147833 1438 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 21:06:47.148648 1410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:47.127477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35254 (local address 127.27.225.254:41039)
0806 21:06:47.127632 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:47.127638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:47.127824 (+ 186us) server_negotiation.cc:407] Connection header received
0806 21:06:47.128034 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:47.128037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:47.128109 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:47.128226 (+ 117us) server_negotiation.cc:226] Negotiated authn=JWT
0806 21:06:47.128479 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:47.129337 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:47.130310 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:47.130674 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:47.130775 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:47.131126 (+ 351us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 21:06:47.148164 (+ 17038us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32815/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 21:06:47.148487 (+ 323us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35254: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32815/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":60}
W20260806 21:06:47.148744 1410 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35254: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32815/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 21:06:47.150238 1437 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:06:47.127348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41039 (local address 127.0.0.1:35254)
0806 21:06:47.127736 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:47.127750 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:47.127865 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:47.128254 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:47.128256 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:47.128269 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0806 21:06:47.128372 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:47.128377 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:47.129473 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:47.129476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:47.130196 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:47.130206 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:47.130797 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:47.130800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:47.130870 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:47.130918 (+ 48us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 21:06:47.148504 (+ 17586us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32815/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 21:06:47.148812 (+ 308us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.27.225.254:41039: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32815/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":123,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.27.225.254:41039: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32815/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 21:06:47.151281 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 1338
2026-08-06T21:06:47Z chronyd exiting
I20260806 21:06:47.190388 28551 test_util.cc:175] -----------------------------------------------
I20260806 21:06:47.190510 28551 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050378084738-28551-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (339 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 21:06:47.191673 28551 mini_oidc.cc:152] Starting JWKS server
I20260806 21:06:47.192579 28551 webserver.cc:460] Webserver started at https://127.0.0.1:41027/ using document root <none> and password file <none>
I20260806 21:06:47.192721 28551 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 21:06:47.192996 28551 webserver.cc:460] Webserver started at http://127.0.0.1:45087/ using document root <none> and password file <none>
2026-08-06T21:06:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:06:47Z Disabled control of system clock
I20260806 21:06:47.203768 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:38587
--webserver_interface=127.27.225.254
--webserver_port=0
--builtin_ntp_servers=127.27.225.212:39283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:38587
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41027/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050378084738-28551-0/testchainca.cert with env {}
W20260806 21:06:47.321204 1444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:47.321512 1444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:47.321558 1444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:47.325726 1444 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:06:47.325809 1444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:47.325855 1444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:47.325872 1444 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:06:47.325888 1444 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 21:06:47.325906 1444 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 21:06:47.325919 1444 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41027/jwks.json
I20260806 21:06:47.331733 1444 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:39283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:38587
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:38587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41027/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050378084738-28551-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:47.332867 1444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:47.333897 1444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:47.343573 1449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:47.343556 1452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:47.343561 1450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:47.343704 1444 server_base.cc:1034] running on GCE node
I20260806 21:06:47.344507 1444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:47.345726 1444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:47.346916 1444 hybrid_clock.cc:648] HybridClock initialized: now 1786050407346887 us; error 41 us; skew 500 ppm
I20260806 21:06:47.348858 1444 webserver.cc:460] Webserver started at http://127.27.225.254:42991/ using document root <none> and password file <none>
I20260806 21:06:47.349411 1444 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:47.349471 1444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:47.349644 1444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:47.351759 1444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "bfd914d933c8473bbbd1e66225e8bb1a"
format_stamp: "Formatted at 2026-08-06 21:06:47 on dist-test-slave-69rg"
server_key: "0ca7d4ec50daf138f9b0e96df20cb40c"
server_key_iv: "00f0ea39db6a0e56e9f8db6c0d281294"
server_key_version: "encryptionkey@0"
I20260806 21:06:47.352272 1444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "bfd914d933c8473bbbd1e66225e8bb1a"
format_stamp: "Formatted at 2026-08-06 21:06:47 on dist-test-slave-69rg"
server_key: "0ca7d4ec50daf138f9b0e96df20cb40c"
server_key_iv: "00f0ea39db6a0e56e9f8db6c0d281294"
server_key_version: "encryptionkey@0"
I20260806 21:06:47.356091 1444 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:06:47.358624 1458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:47.359956 1444 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:06:47.360109 1444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "bfd914d933c8473bbbd1e66225e8bb1a"
format_stamp: "Formatted at 2026-08-06 21:06:47 on dist-test-slave-69rg"
server_key: "0ca7d4ec50daf138f9b0e96df20cb40c"
server_key_iv: "00f0ea39db6a0e56e9f8db6c0d281294"
server_key_version: "encryptionkey@0"
I20260806 21:06:47.360237 1444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:47.379506 1444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:47.380311 1444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:47.380522 1444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:47.388034 1444 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:38587
I20260806 21:06:47.388046 1511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:38587 every 8 connection(s)
I20260806 21:06:47.389099 1444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:06:47.391104 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 1444
I20260806 21:06:47.391237 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:06:47.391554 28551 external_mini_cluster.cc:1428] Setting key 268dfec67af0db12d39ac347d8269e26
I20260806 21:06:47.393136 1512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:47.399992 1512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfd914d933c8473bbbd1e66225e8bb1a: Bootstrap starting.
I20260806 21:06:47.401726 1515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:47.392960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55774 (local address 127.27.225.254:38587)
0806 21:06:47.393660 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:47.393678 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:47.393716 (+ 38us) server_negotiation.cc:407] Connection header received
0806 21:06:47.394454 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:47.394470 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:47.394865 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:47.395161 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:47.395501 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:47.396491 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:47.397737 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:47.398408 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:47.398558 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:47.399605 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:47.399611 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:47.399627 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:06:47.400072 (+ 445us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:47.400242 (+ 170us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:06:47.400284 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:47.400372 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:47.400547 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 21:06:47.400894 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":158,"threads_started":1}
I20260806 21:06:47.402611 1512 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfd914d933c8473bbbd1e66225e8bb1a: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:47.403396 1512 log.cc:824] T 00000000000000000000000000000000 P bfd914d933c8473bbbd1e66225e8bb1a: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:47.405153 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:06:47.405790 1512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfd914d933c8473bbbd1e66225e8bb1a: No bootstrap required, opened a new log
I20260806 21:06:47.409806 1512 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfd914d933c8473bbbd1e66225e8bb1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd914d933c8473bbbd1e66225e8bb1a" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 38587 } }
I20260806 21:06:47.410223 1512 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfd914d933c8473bbbd1e66225e8bb1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:47.410336 1512 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfd914d933c8473bbbd1e66225e8bb1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfd914d933c8473bbbd1e66225e8bb1a, State: Initialized, Role: FOLLOWER
I20260806 21:06:47.410907 1512 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfd914d933c8473bbbd1e66225e8bb1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd914d933c8473bbbd1e66225e8bb1a" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 38587 } }
I20260806 21:06:47.411090 1512 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfd914d933c8473bbbd1e66225e8bb1a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:06:47.411185 1512 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfd914d933c8473bbbd1e66225e8bb1a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:06:47.411329 1512 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfd914d933c8473bbbd1e66225e8bb1a [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:47.412770 1512 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfd914d933c8473bbbd1e66225e8bb1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd914d933c8473bbbd1e66225e8bb1a" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 38587 } }
I20260806 21:06:47.413187 1512 leader_election.cc:302] T 00000000000000000000000000000000 P bfd914d933c8473bbbd1e66225e8bb1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfd914d933c8473bbbd1e66225e8bb1a; no voters:
I20260806 21:06:47.413583 1512 leader_election.cc:288] T 00000000000000000000000000000000 P bfd914d933c8473bbbd1e66225e8bb1a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:06:47.413808 1517 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfd914d933c8473bbbd1e66225e8bb1a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:47.414580 1517 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfd914d933c8473bbbd1e66225e8bb1a [term 1 LEADER]: Becoming Leader. State: Replica: bfd914d933c8473bbbd1e66225e8bb1a, State: Running, Role: LEADER
I20260806 21:06:47.415373 1517 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfd914d933c8473bbbd1e66225e8bb1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd914d933c8473bbbd1e66225e8bb1a" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 38587 } }
I20260806 21:06:47.415835 1512 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfd914d933c8473bbbd1e66225e8bb1a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:06:47.417868 1519 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd914d933c8473bbbd1e66225e8bb1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfd914d933c8473bbbd1e66225e8bb1a. Latest consensus state: current_term: 1 leader_uuid: "bfd914d933c8473bbbd1e66225e8bb1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd914d933c8473bbbd1e66225e8bb1a" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 38587 } } }
I20260806 21:06:47.418007 1519 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd914d933c8473bbbd1e66225e8bb1a [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:47.417794 1518 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfd914d933c8473bbbd1e66225e8bb1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfd914d933c8473bbbd1e66225e8bb1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfd914d933c8473bbbd1e66225e8bb1a" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 38587 } } }
I20260806 21:06:47.418699 1518 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfd914d933c8473bbbd1e66225e8bb1a [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:47.418907 1526 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:06:47.423599 1526 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:06:47.436920 1526 catalog_manager.cc:1347] Generated new cluster ID: d4cfefc53bd04f999cd605f4eb015887
I20260806 21:06:47.437268 1526 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:06:47.450546 1526 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:06:47.451543 1526 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:06:47.472273 1526 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfd914d933c8473bbbd1e66225e8bb1a: Generated new TSK 0
I20260806 21:06:47.473101 1526 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 21:06:47.503909 1542 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 21:06:47.504017 1542 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 21:06:47.504915 1515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:47.480638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55776 (local address 127.27.225.254:38587)
0806 21:06:47.480774 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:47.480780 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:47.480993 (+ 213us) server_negotiation.cc:407] Connection header received
0806 21:06:47.481212 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:47.481215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:47.481263 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:47.481382 (+ 119us) server_negotiation.cc:226] Negotiated authn=JWT
0806 21:06:47.481643 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:47.482250 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:47.483239 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:47.483472 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:47.483590 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:47.483956 (+ 366us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 21:06:47.504374 (+ 20418us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41027/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 21:06:47.504729 (+ 355us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55776: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41027/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 21:06:47.505010 1515 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55776: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41027/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 21:06:47.505030 1541 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:06:47.480561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:38587 (local address 127.0.0.1:55776)
0806 21:06:47.480889 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:47.480902 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:47.481048 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:47.481389 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:47.481392 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:47.481401 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0806 21:06:47.481525 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:47.481531 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:47.482385 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:47.482389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:47.483107 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:47.483116 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:47.483600 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:47.483604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:47.483730 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:47.483756 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 21:06:47.504635 (+ 20879us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41027/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 21:06:47.504833 (+ 198us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.27.225.254:38587: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41027/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":105,"threads_started":1}
I20260806 21:06:47.505744 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 1444
2026-08-06T21:06:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (355 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 21:06:47.547474 28551 mini_oidc.cc:152] Starting JWKS server
I20260806 21:06:47.548391 28551 webserver.cc:460] Webserver started at https://127.0.0.1:41151/ using document root <none> and password file <none>
I20260806 21:06:47.548480 28551 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 21:06:47.548741 28551 webserver.cc:460] Webserver started at http://127.0.0.1:43321/ using document root <none> and password file <none>
2026-08-06T21:06:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:06:47Z Disabled control of system clock
I20260806 21:06:47.559437 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:44773
--webserver_interface=127.27.225.254
--webserver_port=0
--builtin_ntp_servers=127.27.225.212:40979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:44773
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41151/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 21:06:47.673079 1548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:47.673381 1548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:47.673422 1548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:47.677613 1548 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:06:47.677692 1548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:47.677708 1548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:47.677726 1548 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:06:47.677742 1548 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 21:06:47.677757 1548 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 21:06:47.677771 1548 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41151/jwks.json
I20260806 21:06:47.682873 1548 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:40979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:44773
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:44773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41151/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:47.684042 1548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:47.685444 1548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:47.694178 1553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:47.694290 1548 server_base.cc:1034] running on GCE node
W20260806 21:06:47.694168 1554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:47.694167 1556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:47.695009 1548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:47.696170 1548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:47.697396 1548 hybrid_clock.cc:648] HybridClock initialized: now 1786050407697362 us; error 47 us; skew 500 ppm
I20260806 21:06:47.699532 1548 webserver.cc:460] Webserver started at http://127.27.225.254:44281/ using document root <none> and password file <none>
I20260806 21:06:47.700129 1548 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:47.700238 1548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:47.700471 1548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:47.702653 1548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "e83d8b05aefb4f52a05e882b4aa27609"
format_stamp: "Formatted at 2026-08-06 21:06:47 on dist-test-slave-69rg"
server_key: "f49b30c39cf5219e03409418076c08db"
server_key_iv: "25ae92abfef63c095b4f590a0fe0f5f2"
server_key_version: "encryptionkey@0"
I20260806 21:06:47.703223 1548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "e83d8b05aefb4f52a05e882b4aa27609"
format_stamp: "Formatted at 2026-08-06 21:06:47 on dist-test-slave-69rg"
server_key: "f49b30c39cf5219e03409418076c08db"
server_key_iv: "25ae92abfef63c095b4f590a0fe0f5f2"
server_key_version: "encryptionkey@0"
I20260806 21:06:47.707007 1548 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:06:47.709470 1563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:47.710812 1548 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:06:47.710971 1548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "e83d8b05aefb4f52a05e882b4aa27609"
format_stamp: "Formatted at 2026-08-06 21:06:47 on dist-test-slave-69rg"
server_key: "f49b30c39cf5219e03409418076c08db"
server_key_iv: "25ae92abfef63c095b4f590a0fe0f5f2"
server_key_version: "encryptionkey@0"
I20260806 21:06:47.711093 1548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:47.740310 1548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:47.741039 1548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:47.741218 1548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:47.748539 1548 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:44773
I20260806 21:06:47.748548 1615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:44773 every 8 connection(s)
I20260806 21:06:47.749722 1548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:06:47.753015 1616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:47.756315 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 1548
I20260806 21:06:47.756421 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:06:47.756718 28551 external_mini_cluster.cc:1428] Setting key deb11ae9b6df0bb4296abe322d4622f1
I20260806 21:06:47.759043 1616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e83d8b05aefb4f52a05e882b4aa27609: Bootstrap starting.
I20260806 21:06:47.761531 1616 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e83d8b05aefb4f52a05e882b4aa27609: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:47.762534 1616 log.cc:824] T 00000000000000000000000000000000 P e83d8b05aefb4f52a05e882b4aa27609: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:47.764920 1616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e83d8b05aefb4f52a05e882b4aa27609: No bootstrap required, opened a new log
I20260806 21:06:47.766651 1619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:47.758238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38836 (local address 127.27.225.254:44773)
0806 21:06:47.758889 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:47.758905 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:47.758941 (+ 36us) server_negotiation.cc:407] Connection header received
0806 21:06:47.759658 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:47.759676 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:47.760066 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:47.760363 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:47.760728 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:47.761765 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:47.763020 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:47.763675 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:47.763805 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:47.764720 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:47.764725 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:47.764739 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:06:47.765233 (+ 494us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:47.765294 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:06:47.765316 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:47.765396 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:47.765564 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 21:06:47.765890 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":149,"threads_started":1}
I20260806 21:06:47.768770 1616 raft_consensus.cc:348] T 00000000000000000000000000000000 P e83d8b05aefb4f52a05e882b4aa27609 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e83d8b05aefb4f52a05e882b4aa27609" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44773 } }
I20260806 21:06:47.769032 1616 raft_consensus.cc:374] T 00000000000000000000000000000000 P e83d8b05aefb4f52a05e882b4aa27609 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:47.769119 1616 raft_consensus.cc:729] T 00000000000000000000000000000000 P e83d8b05aefb4f52a05e882b4aa27609 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e83d8b05aefb4f52a05e882b4aa27609, State: Initialized, Role: FOLLOWER
I20260806 21:06:47.769611 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:06:47.769639 1616 consensus_queue.cc:260] T 00000000000000000000000000000000 P e83d8b05aefb4f52a05e882b4aa27609 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e83d8b05aefb4f52a05e882b4aa27609" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44773 } }
I20260806 21:06:47.769860 1616 raft_consensus.cc:388] T 00000000000000000000000000000000 P e83d8b05aefb4f52a05e882b4aa27609 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:06:47.769958 1616 raft_consensus.cc:482] T 00000000000000000000000000000000 P e83d8b05aefb4f52a05e882b4aa27609 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:06:47.770063 1616 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e83d8b05aefb4f52a05e882b4aa27609 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:47.771155 1616 raft_consensus.cc:504] T 00000000000000000000000000000000 P e83d8b05aefb4f52a05e882b4aa27609 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e83d8b05aefb4f52a05e882b4aa27609" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44773 } }
I20260806 21:06:47.771509 1616 leader_election.cc:302] T 00000000000000000000000000000000 P e83d8b05aefb4f52a05e882b4aa27609 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e83d8b05aefb4f52a05e882b4aa27609; no voters:
I20260806 21:06:47.771819 1616 leader_election.cc:288] T 00000000000000000000000000000000 P e83d8b05aefb4f52a05e882b4aa27609 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:06:47.771934 1621 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e83d8b05aefb4f52a05e882b4aa27609 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:47.772249 1621 raft_consensus.cc:686] T 00000000000000000000000000000000 P e83d8b05aefb4f52a05e882b4aa27609 [term 1 LEADER]: Becoming Leader. State: Replica: e83d8b05aefb4f52a05e882b4aa27609, State: Running, Role: LEADER
I20260806 21:06:47.772549 1621 consensus_queue.cc:237] T 00000000000000000000000000000000 P e83d8b05aefb4f52a05e882b4aa27609 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e83d8b05aefb4f52a05e882b4aa27609" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44773 } }
I20260806 21:06:47.773105 1616 sys_catalog.cc:564] T 00000000000000000000000000000000 P e83d8b05aefb4f52a05e882b4aa27609 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:06:47.774268 1623 sys_catalog.cc:455] T 00000000000000000000000000000000 P e83d8b05aefb4f52a05e882b4aa27609 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e83d8b05aefb4f52a05e882b4aa27609. Latest consensus state: current_term: 1 leader_uuid: "e83d8b05aefb4f52a05e882b4aa27609" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e83d8b05aefb4f52a05e882b4aa27609" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44773 } } }
I20260806 21:06:47.774277 1622 sys_catalog.cc:455] T 00000000000000000000000000000000 P e83d8b05aefb4f52a05e882b4aa27609 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e83d8b05aefb4f52a05e882b4aa27609" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e83d8b05aefb4f52a05e882b4aa27609" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44773 } } }
I20260806 21:06:47.774415 1622 sys_catalog.cc:458] T 00000000000000000000000000000000 P e83d8b05aefb4f52a05e882b4aa27609 [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:47.774415 1623 sys_catalog.cc:458] T 00000000000000000000000000000000 P e83d8b05aefb4f52a05e882b4aa27609 [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:47.774880 1631 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:06:47.778059 1631 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:06:47.784124 1631 catalog_manager.cc:1347] Generated new cluster ID: a811e08cd90e47898453e63b6d8a6de6
I20260806 21:06:47.784206 1631 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:06:47.799485 1631 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:06:47.800834 1631 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:06:47.809404 1631 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e83d8b05aefb4f52a05e882b4aa27609: Generated new TSK 0
I20260806 21:06:47.810302 1631 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:06:47.826686 1619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:47.821304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38840 (local address 127.27.225.254:44773)
0806 21:06:47.821449 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:47.821455 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:47.821693 (+ 238us) server_negotiation.cc:407] Connection header received
0806 21:06:47.821897 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:47.821900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:47.821956 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:47.822049 (+ 93us) server_negotiation.cc:226] Negotiated authn=JWT
0806 21:06:47.822386 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:47.822959 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:47.824208 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:47.824433 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:47.824504 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:47.824835 (+ 331us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 21:06:47.826315 (+ 1480us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41151/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 21:06:47.826541 (+ 226us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38840: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41151/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":56}
W20260806 21:06:47.826761 1619 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38840: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41151/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 21:06:47.828570 1646 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:06:47.821261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:44773 (local address 127.0.0.1:38840)
0806 21:06:47.821574 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:47.821590 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:47.821712 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:47.822070 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:47.822073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:47.822082 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0806 21:06:47.822217 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:47.822224 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:47.823083 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:47.823087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:47.824081 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:47.824091 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:47.824554 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:47.824558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:47.824645 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:47.824673 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 21:06:47.826481 (+ 1808us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41151/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 21:06:47.828391 (+ 1910us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.27.225.254:44773: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41151/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":1697,"thread_start_us":122,"threads_started":1}
I20260806 21:06:47.829175 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 1548
E20260806 21:06:47.838495 1647 webserver.cc:533] Webserver: sslize failed:
E20260806 21:06:47.838584 1647 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T21:06:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (321 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T21:06:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:06:47Z Disabled control of system clock
I20260806 21:06:47.879940 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:34603
--webserver_interface=127.27.225.254
--webserver_port=0
--builtin_ntp_servers=127.27.225.212:34071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:34603
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050378084738-28551-0/testchainca.cert with env {}
W20260806 21:06:47.994481 1651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:47.994765 1651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:47.994808 1651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:47.998982 1651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:06:47.999048 1651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:47.999064 1651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:47.999082 1651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:06:47.999097 1651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 21:06:47.999151 1651 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 21:06:47.999194 1651 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 21:06:48.004776 1651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:34071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:34603
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:34603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050378084738-28551-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:48.006044 1651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:48.007145 1651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:48.016770 1656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:48.016834 1657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:48.016960 1651 server_base.cc:1034] running on GCE node
W20260806 21:06:48.016788 1659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:48.017735 1651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:48.018976 1651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:48.020218 1651 hybrid_clock.cc:648] HybridClock initialized: now 1786050408020184 us; error 46 us; skew 500 ppm
I20260806 21:06:48.022439 1651 webserver.cc:460] Webserver started at http://127.27.225.254:38555/ using document root <none> and password file <none>
I20260806 21:06:48.023138 1651 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:48.023218 1651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:48.023458 1651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:48.025992 1651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "ef119105477049629d84dfeacdd3b19c"
format_stamp: "Formatted at 2026-08-06 21:06:48 on dist-test-slave-69rg"
server_key: "bf39803c5fec55395b75dafce6ad0680"
server_key_iv: "82c8452585b0df40af70d65c5e78f419"
server_key_version: "encryptionkey@0"
I20260806 21:06:48.026744 1651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "ef119105477049629d84dfeacdd3b19c"
format_stamp: "Formatted at 2026-08-06 21:06:48 on dist-test-slave-69rg"
server_key: "bf39803c5fec55395b75dafce6ad0680"
server_key_iv: "82c8452585b0df40af70d65c5e78f419"
server_key_version: "encryptionkey@0"
I20260806 21:06:48.031281 1651 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:06:48.034221 1665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:48.035593 1651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:06:48.035768 1651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "ef119105477049629d84dfeacdd3b19c"
format_stamp: "Formatted at 2026-08-06 21:06:48 on dist-test-slave-69rg"
server_key: "bf39803c5fec55395b75dafce6ad0680"
server_key_iv: "82c8452585b0df40af70d65c5e78f419"
server_key_version: "encryptionkey@0"
I20260806 21:06:48.035900 1651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:48.067377 1651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:48.068267 1651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:48.068464 1651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:48.077073 1651 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:34603
I20260806 21:06:48.077099 1717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:34603 every 8 connection(s)
I20260806 21:06:48.078152 1651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:06:48.081058 1718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:48.086943 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 1651
I20260806 21:06:48.087093 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:06:48.087283 1718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef119105477049629d84dfeacdd3b19c: Bootstrap starting.
I20260806 21:06:48.087420 28551 external_mini_cluster.cc:1428] Setting key 9513aa1675c67f13715ff0d6cc872caa
I20260806 21:06:48.090212 1718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef119105477049629d84dfeacdd3b19c: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:48.090993 1718 log.cc:824] T 00000000000000000000000000000000 P ef119105477049629d84dfeacdd3b19c: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:48.092921 1718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef119105477049629d84dfeacdd3b19c: No bootstrap required, opened a new log
I20260806 21:06:48.096848 1721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:48.088791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38722 (local address 127.27.225.254:34603)
0806 21:06:48.089557 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:48.089572 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:48.089611 (+ 39us) server_negotiation.cc:407] Connection header received
0806 21:06:48.090426 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:48.090446 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:48.090847 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:48.091206 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:48.091636 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:48.092584 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:48.093520 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:48.094052 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:48.094231 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:48.095134 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:48.095139 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:48.095151 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:06:48.095521 (+ 370us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:48.095575 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:06:48.095593 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:48.095710 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:48.095857 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 21:06:48.096122 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":170,"threads_started":1}
I20260806 21:06:48.096820 1718 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef119105477049629d84dfeacdd3b19c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef119105477049629d84dfeacdd3b19c" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 34603 } }
I20260806 21:06:48.097138 1718 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef119105477049629d84dfeacdd3b19c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:48.097224 1718 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef119105477049629d84dfeacdd3b19c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef119105477049629d84dfeacdd3b19c, State: Initialized, Role: FOLLOWER
I20260806 21:06:48.097651 1718 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef119105477049629d84dfeacdd3b19c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef119105477049629d84dfeacdd3b19c" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 34603 } }
I20260806 21:06:48.097806 1718 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef119105477049629d84dfeacdd3b19c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:06:48.098593 1718 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef119105477049629d84dfeacdd3b19c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:06:48.098691 1718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef119105477049629d84dfeacdd3b19c [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:48.099871 1718 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef119105477049629d84dfeacdd3b19c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef119105477049629d84dfeacdd3b19c" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 34603 } }
I20260806 21:06:48.100107 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:06:48.100281 1718 leader_election.cc:302] T 00000000000000000000000000000000 P ef119105477049629d84dfeacdd3b19c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef119105477049629d84dfeacdd3b19c; no voters:
I20260806 21:06:48.100628 1718 leader_election.cc:288] T 00000000000000000000000000000000 P ef119105477049629d84dfeacdd3b19c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:06:48.100852 1723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef119105477049629d84dfeacdd3b19c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:48.101107 1723 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef119105477049629d84dfeacdd3b19c [term 1 LEADER]: Becoming Leader. State: Replica: ef119105477049629d84dfeacdd3b19c, State: Running, Role: LEADER
I20260806 21:06:48.101650 1723 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef119105477049629d84dfeacdd3b19c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef119105477049629d84dfeacdd3b19c" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 34603 } }
I20260806 21:06:48.102145 1718 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef119105477049629d84dfeacdd3b19c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:06:48.102953 1723 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef119105477049629d84dfeacdd3b19c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef119105477049629d84dfeacdd3b19c. Latest consensus state: current_term: 1 leader_uuid: "ef119105477049629d84dfeacdd3b19c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef119105477049629d84dfeacdd3b19c" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 34603 } } }
I20260806 21:06:48.103080 1723 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef119105477049629d84dfeacdd3b19c [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:48.103440 1725 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef119105477049629d84dfeacdd3b19c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef119105477049629d84dfeacdd3b19c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef119105477049629d84dfeacdd3b19c" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 34603 } } }
I20260806 21:06:48.103559 1725 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef119105477049629d84dfeacdd3b19c [sys.catalog]: This master's current role is: LEADER
W20260806 21:06:48.107569 1739 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ef119105477049629d84dfeacdd3b19c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:06:48.107648 1739 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:06:48.107739 1731 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:06:48.108443 1731 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:06:48.114116 1731 catalog_manager.cc:1347] Generated new cluster ID: 2085bac4550b4bdd8cd286ba2d434265
I20260806 21:06:48.114229 1731 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:06:48.131690 1731 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:06:48.132781 1731 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:06:48.146652 1731 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef119105477049629d84dfeacdd3b19c: Generated new TSK 0
I20260806 21:06:48.147548 1731 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:06:48.606848 1721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:48.163638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38738 (local address 127.27.225.254:34603)
0806 21:06:48.163792 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:48.163799 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:48.163945 (+ 146us) server_negotiation.cc:407] Connection header received
0806 21:06:48.164095 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:48.164099 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:48.164222 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:48.164352 (+ 130us) server_negotiation.cc:226] Negotiated authn=JWT
0806 21:06:48.164571 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:48.165433 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:48.166402 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:48.166768 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:48.166876 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:48.167230 (+ 354us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 21:06:48.606453 (+439223us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 21:06:48.606722 (+ 269us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38738: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 21:06:48.606954 1721 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38738: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 21:06:48.607090 1747 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:06:48.163504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:34603 (local address 127.0.0.1:38738)
0806 21:06:48.163852 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:48.163865 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:48.163970 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:48.164357 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:48.164359 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:48.164366 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0806 21:06:48.164467 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:48.164471 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:48.165549 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:48.165552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:48.166276 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:48.166287 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:48.166913 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:48.166916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:48.166994 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:48.167022 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 21:06:48.606707 (+439685us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 21:06:48.606901 (+ 194us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.27.225.254:34603: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":119,"threads_started":1}
I20260806 21:06:48.607827 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 1651
2026-08-06T21:06:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (758 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (131 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (131 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:06:48 dist-test-slave-69rg krb5kdc[1760](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:06:48 dist-test-slave-69rg krb5kdc[1760](info): set up 2 sockets
Aug 06 21:06:48 dist-test-slave-69rg krb5kdc[1760](info): commencing operation
krb5kdc: starting...
W20260806 21:06:49.943857 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:06:50 dist-test-slave-69rg krb5kdc[1760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:06:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:06:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:06:50.105567 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:44027
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:41027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:44027
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:50.222935 1774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:50.223234 1774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:50.223278 1774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:50.227566 1774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:06:50.227645 1774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:50.227666 1774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:50.227690 1774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:06:50.227707 1774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:06:50.233791 1774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:41027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:44027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:44027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:50.234932 1774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:50.235917 1774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:06:50.244903 1774 server_base.cc:1034] running on GCE node
W20260806 21:06:50.244882 1779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:50.244848 1782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:50.244846 1780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:50.245802 1774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:50.247069 1774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:50.248260 1774 hybrid_clock.cc:648] HybridClock initialized: now 1786050410248238 us; error 35 us; skew 500 ppm
Aug 06 21:06:50 dist-test-slave-69rg krb5kdc[1760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050410, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:50.251392 1774 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:06:50.252383 1774 webserver.cc:460] Webserver started at http://127.27.225.254:45337/ using document root <none> and password file <none>
I20260806 21:06:50.252969 1774 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:50.253067 1774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:50.253310 1774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:50.255539 1774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "952b896b385b4fcb815127a5c52ef9d1"
format_stamp: "Formatted at 2026-08-06 21:06:50 on dist-test-slave-69rg"
server_key: "66390e91dd8a45f84478d3057401dc09"
server_key_iv: "77c2b271aee0be3aa72b3976ebd32acd"
server_key_version: "encryptionkey@0"
I20260806 21:06:50.256145 1774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "952b896b385b4fcb815127a5c52ef9d1"
format_stamp: "Formatted at 2026-08-06 21:06:50 on dist-test-slave-69rg"
server_key: "66390e91dd8a45f84478d3057401dc09"
server_key_iv: "77c2b271aee0be3aa72b3976ebd32acd"
server_key_version: "encryptionkey@0"
I20260806 21:06:50.260186 1774 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:06:50.262887 1789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:50.264367 1774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 21:06:50.264537 1774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "952b896b385b4fcb815127a5c52ef9d1"
format_stamp: "Formatted at 2026-08-06 21:06:50 on dist-test-slave-69rg"
server_key: "66390e91dd8a45f84478d3057401dc09"
server_key_iv: "77c2b271aee0be3aa72b3976ebd32acd"
server_key_version: "encryptionkey@0"
I20260806 21:06:50.264691 1774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:50.274387 1774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:50.278091 1774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:50.278322 1774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:50.286677 1774 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:44027
I20260806 21:06:50.286793 1842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:44027 every 8 connection(s)
I20260806 21:06:50.287712 1774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:06:50.290614 1843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:50.293274 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 1774
I20260806 21:06:50.293440 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:06:50.293797 28551 external_mini_cluster.cc:1428] Setting key 4c1324bbf7a06fd26e52f92f5e2bf623
I20260806 21:06:50.297243 1843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 952b896b385b4fcb815127a5c52ef9d1: Bootstrap starting.
I20260806 21:06:50.299719 1843 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 952b896b385b4fcb815127a5c52ef9d1: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:50.300541 1843 log.cc:824] T 00000000000000000000000000000000 P 952b896b385b4fcb815127a5c52ef9d1: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:50.302693 1843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 952b896b385b4fcb815127a5c52ef9d1: No bootstrap required, opened a new log
Aug 06 21:06:50 dist-test-slave-69rg krb5kdc[1760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:50.306064 1843 raft_consensus.cc:348] T 00000000000000000000000000000000 P 952b896b385b4fcb815127a5c52ef9d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "952b896b385b4fcb815127a5c52ef9d1" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44027 } }
I20260806 21:06:50.306355 1843 raft_consensus.cc:374] T 00000000000000000000000000000000 P 952b896b385b4fcb815127a5c52ef9d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:50.306442 1843 raft_consensus.cc:729] T 00000000000000000000000000000000 P 952b896b385b4fcb815127a5c52ef9d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 952b896b385b4fcb815127a5c52ef9d1, State: Initialized, Role: FOLLOWER
I20260806 21:06:50.306958 1843 consensus_queue.cc:260] T 00000000000000000000000000000000 P 952b896b385b4fcb815127a5c52ef9d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "952b896b385b4fcb815127a5c52ef9d1" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44027 } }
I20260806 21:06:50.307119 1843 raft_consensus.cc:388] T 00000000000000000000000000000000 P 952b896b385b4fcb815127a5c52ef9d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:06:50.307269 1843 raft_consensus.cc:482] T 00000000000000000000000000000000 P 952b896b385b4fcb815127a5c52ef9d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:06:50.307390 1843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 952b896b385b4fcb815127a5c52ef9d1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:50.308707 1843 raft_consensus.cc:504] T 00000000000000000000000000000000 P 952b896b385b4fcb815127a5c52ef9d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "952b896b385b4fcb815127a5c52ef9d1" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44027 } }
I20260806 21:06:50.309104 1843 leader_election.cc:302] T 00000000000000000000000000000000 P 952b896b385b4fcb815127a5c52ef9d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 952b896b385b4fcb815127a5c52ef9d1; no voters:
I20260806 21:06:50.309463 1843 leader_election.cc:288] T 00000000000000000000000000000000 P 952b896b385b4fcb815127a5c52ef9d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:06:50.309620 1848 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 952b896b385b4fcb815127a5c52ef9d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:50.309886 1848 raft_consensus.cc:686] T 00000000000000000000000000000000 P 952b896b385b4fcb815127a5c52ef9d1 [term 1 LEADER]: Becoming Leader. State: Replica: 952b896b385b4fcb815127a5c52ef9d1, State: Running, Role: LEADER
I20260806 21:06:50.310153 1846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:50.295688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35498 (local address 127.27.225.254:44027)
0806 21:06:50.296172 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:50.296194 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:50.296231 (+ 37us) server_negotiation.cc:407] Connection header received
0806 21:06:50.296862 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:50.296886 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:50.297225 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:50.297610 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:50.298529 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:50.299549 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:50.300495 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:50.301035 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:50.301222 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:50.304099 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:50.304105 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:50.304120 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:50.304156 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:50.306721 (+ 2565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:50.307247 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:50.307253 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:50.307259 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:50.307426 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:50.307742 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:50.307748 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:50.307749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:50.308282 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:50.308506 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:50.308726 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 21:06:50.309056 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":176,"threads_started":1}
I20260806 21:06:50.310284 1848 consensus_queue.cc:237] T 00000000000000000000000000000000 P 952b896b385b4fcb815127a5c52ef9d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "952b896b385b4fcb815127a5c52ef9d1" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44027 } }
I20260806 21:06:50.310832 1843 sys_catalog.cc:564] T 00000000000000000000000000000000 P 952b896b385b4fcb815127a5c52ef9d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:06:50.312057 1849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 952b896b385b4fcb815127a5c52ef9d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "952b896b385b4fcb815127a5c52ef9d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "952b896b385b4fcb815127a5c52ef9d1" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44027 } } }
I20260806 21:06:50.312371 1850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 952b896b385b4fcb815127a5c52ef9d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 952b896b385b4fcb815127a5c52ef9d1. Latest consensus state: current_term: 1 leader_uuid: "952b896b385b4fcb815127a5c52ef9d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "952b896b385b4fcb815127a5c52ef9d1" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44027 } } }
I20260806 21:06:50.312500 1849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 952b896b385b4fcb815127a5c52ef9d1 [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:50.313902 1850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 952b896b385b4fcb815127a5c52ef9d1 [sys.catalog]: This master's current role is: LEADER
W20260806 21:06:50.315685 1863 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 952b896b385b4fcb815127a5c52ef9d1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:06:50.315798 1863 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:06:50.315876 1865 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:06:50.316583 1865 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:06:50.322969 1865 catalog_manager.cc:1347] Generated new cluster ID: 3323435190714580b53ae2482d9c0af6
I20260806 21:06:50.323065 1865 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:06:50.342334 1865 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:06:50.343645 1865 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:06:50.362723 1865 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 952b896b385b4fcb815127a5c52ef9d1: Generated new TSK 0
I20260806 21:06:50.363633 1865 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.225.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
I20260806 21:06:50.405892 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:44027
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:41027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:50.527225 1870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:50.527505 1870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:50.527550 1870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:50.531931 1870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:50.532014 1870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:50.532078 1870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:06:50.537569 1870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:41027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:44027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:50.538759 1870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:50.539758 1870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:50.549438 1878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:50.549412 1876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:50.549405 1875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:50.549842 1870 server_base.cc:1034] running on GCE node
I20260806 21:06:50.550388 1870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:50.551064 1870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:50.552265 1870 hybrid_clock.cc:648] HybridClock initialized: now 1786050410552245 us; error 33 us; skew 500 ppm
Aug 06 21:06:50 dist-test-slave-69rg krb5kdc[1760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050410, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:50.555348 1870 init.cc:377] Logged in from keytab as kudu/127.27.225.193@KRBTEST.COM (short username kudu)
I20260806 21:06:50.556298 1870 webserver.cc:460] Webserver started at http://127.27.225.193:45409/ using document root <none> and password file <none>
I20260806 21:06:50.556874 1870 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:50.556972 1870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:50.557214 1870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:50.559405 1870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "d40a567dd3bd48acac188c2ee6409370"
format_stamp: "Formatted at 2026-08-06 21:06:50 on dist-test-slave-69rg"
server_key: "19bcb869a1670093626f55e718eb86bb"
server_key_iv: "1cc42c18303a706aae95221dda2ac429"
server_key_version: "encryptionkey@0"
I20260806 21:06:50.560009 1870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "d40a567dd3bd48acac188c2ee6409370"
format_stamp: "Formatted at 2026-08-06 21:06:50 on dist-test-slave-69rg"
server_key: "19bcb869a1670093626f55e718eb86bb"
server_key_iv: "1cc42c18303a706aae95221dda2ac429"
server_key_version: "encryptionkey@0"
I20260806 21:06:50.564035 1870 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:06:50.566874 1885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:50.568377 1870 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:06:50.568544 1870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "d40a567dd3bd48acac188c2ee6409370"
format_stamp: "Formatted at 2026-08-06 21:06:50 on dist-test-slave-69rg"
server_key: "19bcb869a1670093626f55e718eb86bb"
server_key_iv: "1cc42c18303a706aae95221dda2ac429"
server_key_version: "encryptionkey@0"
I20260806 21:06:50.568687 1870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:50.580657 1870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:50.584314 1870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:50.584539 1870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:50.585175 1870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:50.586110 1870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:50.586186 1870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:50.586258 1870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:50.586303 1870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:50.596196 1870 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:37439
I20260806 21:06:50.596282 2000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:37439 every 8 connection(s)
I20260806 21:06:50.597210 1870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:06:50.605230 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 1870
I20260806 21:06:50.605350 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:06:50.605644 28551 external_mini_cluster.cc:1428] Setting key 339692438b4d2ab948457fcd32c1ac91
Aug 06 21:06:50 dist-test-slave-69rg krb5kdc[1760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050410, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:50.613431 1846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:50.599249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:44625 (local address 127.27.225.254:44027)
0806 21:06:50.599437 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:50.599443 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:50.600317 (+ 874us) server_negotiation.cc:407] Connection header received
0806 21:06:50.601684 (+ 1367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:50.601687 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:50.601806 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:50.601929 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:50.603377 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:50.603961 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:50.605530 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:50.605901 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:50.606069 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:50.609110 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:50.609116 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:50.609123 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:50.609159 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:50.610861 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:50.611574 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:50.611577 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:50.611578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:50.611654 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:50.612091 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:50.612093 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:50.612094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:50.612335 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:50.612422 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:50.613171 (+ 749us) server_negotiation.cc:299] Negotiation successful
0806 21:06:50.613300 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 21:06:50.614846 2003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:50.599648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:44027 (local address 127.27.225.193:44625)
0806 21:06:50.600133 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:50.600191 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:50.601469 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:50.602138 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:50.602146 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:50.602641 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:50.603205 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:50.603220 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:50.604122 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:50.604125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:50.605287 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:50.605337 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:50.606128 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:50.606133 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:50.606336 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:50.607042 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:50.607066 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:50.608917 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:50.610993 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:50.611002 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:50.611022 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:50.611476 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:50.611782 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:50.611788 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:50.611793 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:50.612000 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:50.612476 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:50.612486 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:50.612888 (+ 402us) client_negotiation.cc:769] Sending connection context
0806 21:06:50.613135 (+ 247us) client_negotiation.cc:240] Negotiation successful
0806 21:06:50.613550 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":123,"threads_started":1}
I20260806 21:06:50.616297 2001 heartbeater.cc:347] Connected to a master server at 127.27.225.254:44027
I20260806 21:06:50.616566 2001 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.27.225.194@KRBTEST.COM; defaulting to no policy
I20260806 21:06:50.617129 2001 heartbeater.cc:511] Master 127.27.225.254:44027 requested a full tablet report, sending...
I20260806 21:06:50.618849 1806 ts_manager.cc:194] Registered new tserver with Master: d40a567dd3bd48acac188c2ee6409370 (127.27.225.193:37439)
I20260806 21:06:50.620078 1806 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.193@KRBTEST.COM'} at 127.27.225.193:44625
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
I20260806 21:06:50.642964 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:44027
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:41027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:50.761488 2006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:50.761873 2006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:50.761987 2006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:50.766376 2006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:50.766484 2006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:50.766587 2006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:06:50.772532 2006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:41027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:44027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:50.773777 2006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:50.774883 2006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:50.784907 2011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:50.785051 2014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:50.784895 2012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:50.785324 2006 server_base.cc:1034] running on GCE node
I20260806 21:06:50.785791 2006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:50.786526 2006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:50.787705 2006 hybrid_clock.cc:648] HybridClock initialized: now 1786050410787691 us; error 26 us; skew 500 ppm
Aug 06 21:06:50 dist-test-slave-69rg krb5kdc[1760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050410, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:50.790737 2006 init.cc:377] Logged in from keytab as kudu/127.27.225.194@KRBTEST.COM (short username kudu)
I20260806 21:06:50.791702 2006 webserver.cc:460] Webserver started at http://127.27.225.194:39489/ using document root <none> and password file <none>
I20260806 21:06:50.792313 2006 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:50.792419 2006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:50.792671 2006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:50.794936 2006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "56b5f34015c946e0996b835509d026a1"
format_stamp: "Formatted at 2026-08-06 21:06:50 on dist-test-slave-69rg"
server_key: "f5c4c17764e16e2f183e2f5c1089dff8"
server_key_iv: "9c8106636bc5baa0effc28a5de815c8c"
server_key_version: "encryptionkey@0"
I20260806 21:06:50.795560 2006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "56b5f34015c946e0996b835509d026a1"
format_stamp: "Formatted at 2026-08-06 21:06:50 on dist-test-slave-69rg"
server_key: "f5c4c17764e16e2f183e2f5c1089dff8"
server_key_iv: "9c8106636bc5baa0effc28a5de815c8c"
server_key_version: "encryptionkey@0"
I20260806 21:06:50.799750 2006 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:06:50.802528 2021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:50.803902 2006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:06:50.804055 2006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "56b5f34015c946e0996b835509d026a1"
format_stamp: "Formatted at 2026-08-06 21:06:50 on dist-test-slave-69rg"
server_key: "f5c4c17764e16e2f183e2f5c1089dff8"
server_key_iv: "9c8106636bc5baa0effc28a5de815c8c"
server_key_version: "encryptionkey@0"
I20260806 21:06:50.804179 2006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:50.820888 2006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:50.824134 2006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:50.824368 2006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:50.824990 2006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:50.825911 2006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:50.825979 2006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:50.826046 2006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:50.826093 2006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:50.835829 2006 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:46547
I20260806 21:06:50.835851 2135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:46547 every 8 connection(s)
I20260806 21:06:50.836755 2006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:06:50.841342 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 2006
I20260806 21:06:50.841475 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:06:50.841843 28551 external_mini_cluster.cc:1428] Setting key dfeeeb5d4ecb4405321405763aa3f5d2
Aug 06 21:06:50 dist-test-slave-69rg krb5kdc[1760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050410, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:50.852684 1846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:50.838826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:56593 (local address 127.27.225.254:44027)
0806 21:06:50.838993 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:50.838997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:50.839823 (+ 826us) server_negotiation.cc:407] Connection header received
0806 21:06:50.841293 (+ 1470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:50.841297 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:50.841364 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:50.841477 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:50.843025 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:50.843663 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:50.844836 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:50.845065 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:50.845180 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:50.848144 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:50.848151 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:50.848157 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:50.848195 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:50.850049 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:50.850814 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:50.850817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:50.850818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:50.850901 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:50.851358 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:50.851361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:50.851362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:50.851577 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:50.851674 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:50.852415 (+ 741us) server_negotiation.cc:299] Negotiation successful
0806 21:06:50.852545 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
WARNING: no policy specified for kudu/127.27.225.195@KRBTEST.COM; defaulting to no policy
I20260806 21:06:50.854004 2138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:50.839165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:44027 (local address 127.27.225.194:56593)
0806 21:06:50.839660 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:50.839704 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:50.841020 (+ 1316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:50.841734 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:50.841744 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:50.842274 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:50.842866 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:50.842886 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:50.843813 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:50.843816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:50.844702 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:50.844712 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:50.845225 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:50.845230 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:50.845411 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:50.846081 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:50.846098 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:50.847901 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:50.850204 (+ 2303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:50.850215 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:50.850238 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:50.850683 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:50.851009 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:50.851012 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:50.851014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:50.851230 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:50.851703 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:50.851713 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:50.852089 (+ 376us) client_negotiation.cc:769] Sending connection context
0806 21:06:50.852345 (+ 256us) client_negotiation.cc:240] Negotiation successful
0806 21:06:50.852736 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":115,"threads_started":1}
I20260806 21:06:50.855613 2136 heartbeater.cc:347] Connected to a master server at 127.27.225.254:44027
I20260806 21:06:50.855856 2136 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:50.856418 2136 heartbeater.cc:511] Master 127.27.225.254:44027 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.195@KRBTEST.COM" created.
I20260806 21:06:50.857604 1806 ts_manager.cc:194] Registered new tserver with Master: 56b5f34015c946e0996b835509d026a1 (127.27.225.194:46547)
I20260806 21:06:50.858296 1806 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.194@KRBTEST.COM'} at 127.27.225.194:56593
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
I20260806 21:06:50.873557 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:44027
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:41027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:50.986219 2141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:50.986550 2141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:50.986629 2141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:50.990844 2141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:50.990950 2141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:50.991045 2141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:06:50.996610 2141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:41027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:44027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:50.997881 2141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:50.999017 2141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:51.009222 2149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:51.009222 2147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:51.009212 2146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:51.009662 2141 server_base.cc:1034] running on GCE node
I20260806 21:06:51.010202 2141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:51.010984 2141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:51.012240 2141 hybrid_clock.cc:648] HybridClock initialized: now 1786050411012135 us; error 46 us; skew 500 ppm
Aug 06 21:06:51 dist-test-slave-69rg krb5kdc[1760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050411, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:51.015573 2141 init.cc:377] Logged in from keytab as kudu/127.27.225.195@KRBTEST.COM (short username kudu)
I20260806 21:06:51.016669 2141 webserver.cc:460] Webserver started at http://127.27.225.195:37935/ using document root <none> and password file <none>
I20260806 21:06:51.017323 2141 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:51.017428 2141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:51.017710 2141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:51.020138 2141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "59358a0da36a448b8bed29931be924e0"
format_stamp: "Formatted at 2026-08-06 21:06:51 on dist-test-slave-69rg"
server_key: "7dab6834a76be8fa232e963bc4471a89"
server_key_iv: "4063de8f579966c14b32b719846cce08"
server_key_version: "encryptionkey@0"
I20260806 21:06:51.020793 2141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "59358a0da36a448b8bed29931be924e0"
format_stamp: "Formatted at 2026-08-06 21:06:51 on dist-test-slave-69rg"
server_key: "7dab6834a76be8fa232e963bc4471a89"
server_key_iv: "4063de8f579966c14b32b719846cce08"
server_key_version: "encryptionkey@0"
I20260806 21:06:51.025071 2141 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260806 21:06:51.027916 2157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:51.029244 2141 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 21:06:51.029395 2141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "59358a0da36a448b8bed29931be924e0"
format_stamp: "Formatted at 2026-08-06 21:06:51 on dist-test-slave-69rg"
server_key: "7dab6834a76be8fa232e963bc4471a89"
server_key_iv: "4063de8f579966c14b32b719846cce08"
server_key_version: "encryptionkey@0"
I20260806 21:06:51.029516 2141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:51.044363 2141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:51.048229 2141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:51.048486 2141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:51.049152 2141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:51.050127 2141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:51.050207 2141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:51.050279 2141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:51.050330 2141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:51.059589 2141 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:34929
I20260806 21:06:51.059624 2272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:34929 every 8 connection(s)
I20260806 21:06:51.060521 2141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:06:51.061503 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 2141
I20260806 21:06:51.061641 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:06:51.062007 28551 external_mini_cluster.cc:1428] Setting key 5781421e8d41c2d00904bc11ee6d30a3
Aug 06 21:06:51 dist-test-slave-69rg krb5kdc[1760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050411, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:06:51.075457 1846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:51.063105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:42555 (local address 127.27.225.254:44027)
0806 21:06:51.063258 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:51.063263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:51.063757 (+ 494us) server_negotiation.cc:407] Connection header received
0806 21:06:51.065334 (+ 1577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:51.065337 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:51.065383 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:51.065449 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:51.066654 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:51.067204 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:51.068381 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:51.068620 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:51.068707 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:51.071481 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:51.071485 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:51.071487 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:51.071515 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:51.072984 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:51.073637 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:51.073640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:51.073641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:51.073723 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:51.074195 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:51.074198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:51.074199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:51.074402 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:51.074490 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:51.075125 (+ 635us) server_negotiation.cc:299] Negotiation successful
0806 21:06:51.075288 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 21:06:51.076363 2275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:51.063009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:44027 (local address 127.27.225.195:42555)
0806 21:06:51.063588 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:51.063660 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:51.065150 (+ 1490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:51.065582 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:51.065590 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:51.066036 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:51.066521 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:51.066534 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:51.067315 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:51.067318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:51.068250 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:51.068260 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:51.068769 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:51.068772 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:51.068921 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:51.069683 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:51.069719 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:51.071307 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:51.073157 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:51.073163 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:51.073176 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:51.073517 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:51.073865 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:51.073871 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:51.073873 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:51.074024 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:51.074503 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:51.074508 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:51.074802 (+ 294us) client_negotiation.cc:769] Sending connection context
0806 21:06:51.075003 (+ 201us) client_negotiation.cc:240] Negotiation successful
0806 21:06:51.075368 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":135,"threads_started":1}
I20260806 21:06:51.077554 2273 heartbeater.cc:347] Connected to a master server at 127.27.225.254:44027
I20260806 21:06:51.077862 2273 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:51.078424 2273 heartbeater.cc:511] Master 127.27.225.254:44027 requested a full tablet report, sending...
I20260806 21:06:51.079463 1806 ts_manager.cc:194] Registered new tserver with Master: 59358a0da36a448b8bed29931be924e0 (127.27.225.195:34929)
I20260806 21:06:51.080053 1806 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.195@KRBTEST.COM'} at 127.27.225.195:42555
I20260806 21:06:51.088923 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/test-admin.keytab.
Aug 06 21:06:51 dist-test-slave-69rg krb5kdc[1760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:51.108055 28551 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 21:06:51.111291 2280 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:06:51.109695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49852 (local address 127.27.225.193:37439)
0806 21:06:51.110042 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:51.110057 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:51.110081 (+ 24us) server_negotiation.cc:407] Connection header received
0806 21:06:51.110174 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:51.110179 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:51.110436 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:51.110557 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:51.111072 (+ 515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49852: BlockingRecv error: recv got EOF from 127.0.0.1:49852 (error 108)
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":119,"threads_started":1}
W20260806 21:06:51.111511 28551 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/krb5cc)
Aug 06 21:06:51 dist-test-slave-69rg krb5kdc[1760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:51.112684 28551 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 21:06:51 dist-test-slave-69rg krb5kdc[1760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:51.113646 28551 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 21:06:51 dist-test-slave-69rg krb5kdc[1760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.193@KRBTEST.COM
I20260806 21:06:51.124089 2280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:51.114749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49868 (local address 127.27.225.193:37439)
0806 21:06:51.114881 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:51.114886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:51.114962 (+ 76us) server_negotiation.cc:407] Connection header received
0806 21:06:51.115163 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:51.115165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:51.115205 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:51.115303 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:51.115890 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:51.116656 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:51.117495 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:51.117951 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:51.118079 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:51.120386 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:51.120397 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:51.120411 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:51.120436 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:51.122319 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:51.122744 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:51.122749 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:51.122750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:51.122827 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:51.123106 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:51.123109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:51.123111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:51.123387 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:51.123555 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:51.123819 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 21:06:51.123961 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260806 21:06:51.128530 2280 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:06:51.127407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49882 (local address 127.27.225.193:37439)
0806 21:06:51.127566 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:51.127571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:51.127707 (+ 136us) server_negotiation.cc:407] Connection header received
0806 21:06:51.127849 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:51.127851 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:51.127892 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:51.127951 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:51.128433 (+ 482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49882: BlockingRecv error: recv got EOF from 127.0.0.1:49882 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 21:06:51.128643 28551 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/krb5cc' of type FILE
Aug 06 21:06:51 dist-test-slave-69rg krb5kdc[1760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:51.129801 28551 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 21:06:51 dist-test-slave-69rg krb5kdc[1760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:51.130823 28551 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 21:06:51 dist-test-slave-69rg krb5kdc[1760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.193@KRBTEST.COM
I20260806 21:06:51.140571 2280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:51.131805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49894 (local address 127.27.225.193:37439)
0806 21:06:51.131945 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:51.131950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:51.132056 (+ 106us) server_negotiation.cc:407] Connection header received
0806 21:06:51.132260 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:51.132268 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:51.132309 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:51.132391 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:51.133044 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:51.133559 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:51.134537 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:51.134806 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:51.134897 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:51.137398 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:51.137402 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:51.137404 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:51.137429 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:51.139034 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:51.139491 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:51.139494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:51.139495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:51.139574 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:51.139828 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:51.139830 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:51.139831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:51.139996 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:51.140097 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:51.140337 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 21:06:51.140446 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260806 21:06:51.143806 2280 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:06:51.142729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49898 (local address 127.27.225.193:37439)
0806 21:06:51.142894 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:51.142898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:51.143017 (+ 119us) server_negotiation.cc:407] Connection header received
0806 21:06:51.143165 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:51.143168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:51.143215 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:51.143302 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:51.143700 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49898: BlockingRecv error: recv got EOF from 127.0.0.1:49898 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260806 21:06:51.143944 28551 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050378084738-28551-0/krb5kdc/krb5cc)
Aug 06 21:06:51 dist-test-slave-69rg krb5kdc[1760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:51.145012 28551 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 21:06:51 dist-test-slave-69rg krb5kdc[1760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:51.146016 28551 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 21:06:51 dist-test-slave-69rg krb5kdc[1760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.193@KRBTEST.COM
I20260806 21:06:51.155535 2280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:51.146903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49914 (local address 127.27.225.193:37439)
0806 21:06:51.147033 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:51.147037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:51.147177 (+ 140us) server_negotiation.cc:407] Connection header received
0806 21:06:51.147311 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:51.147318 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:51.147355 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:51.147446 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:51.147979 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:51.148488 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:51.149334 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:51.149552 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:51.149664 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:51.152151 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:51.152155 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:51.152157 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:51.152181 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:51.153456 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:51.153949 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:51.153951 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:51.153953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:51.154029 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:51.154394 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:51.154397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:51.154402 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:51.154606 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:51.154766 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:51.155275 (+ 509us) server_negotiation.cc:299] Negotiation successful
0806 21:06:51.155415 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 21:06:51.156606 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 1870
I20260806 21:06:51.162994 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 2006
I20260806 21:06:51.169492 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 2141
I20260806 21:06:51.176276 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 1774
2026-08-06T21:06:51Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2307 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:06:51 dist-test-slave-69rg krb5kdc[2293](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:06:51 dist-test-slave-69rg krb5kdc[2293](info): set up 2 sockets
Aug 06 21:06:51 dist-test-slave-69rg krb5kdc[2293](info): commencing operation
krb5kdc: starting...
W20260806 21:06:52.261648 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:06:52 dist-test-slave-69rg krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:06:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:06:52Z Disabled control of system clock
WARNING: no policy specified for oryx/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.254.keytab.
Entry for principal oryx/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.254.keytab.
I20260806 21:06:52.397578 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:45179
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.254.keytab
--principal=oryx/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:46765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:45179
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:52.511492 2306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:52.511863 2306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:52.511951 2306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:52.516100 2306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:06:52.516219 2306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:52.516260 2306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:52.516299 2306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:06:52.516335 2306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:06:52.521878 2306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:46765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:45179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.254.keytab
--principal=oryx/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:45179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:52.523052 2306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:52.524107 2306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:52.532820 2314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:52.532810 2311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:52.532994 2306 server_base.cc:1034] running on GCE node
W20260806 21:06:52.532814 2312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:52.533704 2306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:52.534891 2306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:52.536091 2306 hybrid_clock.cc:648] HybridClock initialized: now 1786050412536065 us; error 41 us; skew 500 ppm
Aug 06 21:06:52 dist-test-slave-69rg krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050412, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:52.539002 2306 init.cc:377] Logged in from keytab as oryx/127.27.225.254@KRBTEST.COM (short username oryx)
I20260806 21:06:52.539920 2306 webserver.cc:460] Webserver started at http://127.27.225.254:46361/ using document root <none> and password file <none>
I20260806 21:06:52.540441 2306 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:52.540501 2306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:52.540679 2306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:52.542783 2306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "00de4fec8340498da672d706bcc37acf"
format_stamp: "Formatted at 2026-08-06 21:06:52 on dist-test-slave-69rg"
server_key: "358eada4996bb407847a299c9d42b744"
server_key_iv: "e986b787012557ab6559259cead45ab9"
server_key_version: "encryptionkey@0"
I20260806 21:06:52.543301 2306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "00de4fec8340498da672d706bcc37acf"
format_stamp: "Formatted at 2026-08-06 21:06:52 on dist-test-slave-69rg"
server_key: "358eada4996bb407847a299c9d42b744"
server_key_iv: "e986b787012557ab6559259cead45ab9"
server_key_version: "encryptionkey@0"
I20260806 21:06:52.547147 2306 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:06:52.549681 2321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:52.551023 2306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:06:52.551142 2306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "00de4fec8340498da672d706bcc37acf"
format_stamp: "Formatted at 2026-08-06 21:06:52 on dist-test-slave-69rg"
server_key: "358eada4996bb407847a299c9d42b744"
server_key_iv: "e986b787012557ab6559259cead45ab9"
server_key_version: "encryptionkey@0"
I20260806 21:06:52.551244 2306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:52.575603 2306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:52.578563 2306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:52.578723 2306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:52.586091 2306 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:45179
I20260806 21:06:52.586395 2376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:45179 every 8 connection(s)
I20260806 21:06:52.587194 2306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:06:52.589919 2377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:52.595468 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 2306
I20260806 21:06:52.595582 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:06:52.595773 2377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf: Bootstrap starting.
I20260806 21:06:52.596028 28551 external_mini_cluster.cc:1428] Setting key 1fa4878eb3419e2dae5003b6b7689d6e
I20260806 21:06:52.598500 2377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:52.599323 2377 log.cc:824] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:52.601332 2377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf: No bootstrap required, opened a new log
Aug 06 21:06:52 dist-test-slave-69rg krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM
I20260806 21:06:52.605991 2377 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } }
I20260806 21:06:52.606256 2377 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:52.606338 2377 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00de4fec8340498da672d706bcc37acf, State: Initialized, Role: FOLLOWER
I20260806 21:06:52.606854 2377 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } }
I20260806 21:06:52.607048 2377 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:06:52.607126 2377 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:06:52.607234 2377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:52.608235 2377 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } }
I20260806 21:06:52.608606 2377 leader_election.cc:302] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00de4fec8340498da672d706bcc37acf; no voters:
I20260806 21:06:52.608985 2377 leader_election.cc:288] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:06:52.609088 2382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:52.609264 2382 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [term 1 LEADER]: Becoming Leader. State: Replica: 00de4fec8340498da672d706bcc37acf, State: Running, Role: LEADER
I20260806 21:06:52.609656 2382 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } }
I20260806 21:06:52.610415 2377 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:06:52.611382 2384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00de4fec8340498da672d706bcc37acf. Latest consensus state: current_term: 1 leader_uuid: "00de4fec8340498da672d706bcc37acf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } }
I20260806 21:06:52.611510 2384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:52.611862 2380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:52.597493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52546 (local address 127.27.225.254:45179)
0806 21:06:52.597969 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:52.597987 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:52.598022 (+ 35us) server_negotiation.cc:407] Connection header received
0806 21:06:52.598748 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:52.598767 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:52.599088 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:52.599367 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:52.600322 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:52.601329 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:52.602593 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:52.603064 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:52.603195 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:52.606200 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:52.606205 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:52.606219 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:52.606247 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:52.608267 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:52.608749 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:52.608755 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:52.608762 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:52.608898 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:52.609219 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:52.609222 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:52.609224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:52.609874 (+ 650us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:52.610084 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:52.610394 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 21:06:52.610739 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":163,"threads_started":1}
I20260806 21:06:52.611862 2383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00de4fec8340498da672d706bcc37acf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } }
I20260806 21:06:52.612207 2383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [sys.catalog]: This master's current role is: LEADER
W20260806 21:06:52.615635 2398 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:06:52.615720 2398 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:06:52.615808 2391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:06:52.616472 2391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:06:52.622035 2391 catalog_manager.cc:1347] Generated new cluster ID: 42ffcff42f5a4ac2a89ff0d847fddd0b
I20260806 21:06:52.622116 2391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:06:52.641288 2391 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:06:52.642660 2391 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:06:52.648594 2391 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf: Generated new TSK 0
I20260806 21:06:52.649469 2391 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.27.225.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.225.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.225.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.193.keytab.
Entry for principal oryx/127.27.225.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.193.keytab.
I20260806 21:06:52.686865 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:45179
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.193.keytab
--principal=oryx/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:46765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:52.804718 2403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:52.804991 2403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:52.805037 2403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:52.809089 2403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:52.809158 2403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:52.809219 2403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:06:52.815106 2403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:46765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.193.keytab
--principal=oryx/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:45179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:52.816161 2403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:52.817158 2403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:52.826903 2409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:52.826932 2408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:52.826964 2411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:52.827349 2403 server_base.cc:1034] running on GCE node
I20260806 21:06:52.827765 2403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:52.828416 2403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:52.829607 2403 hybrid_clock.cc:648] HybridClock initialized: now 1786050412829599 us; error 33 us; skew 500 ppm
Aug 06 21:06:52 dist-test-slave-69rg krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050412, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:52.832763 2403 init.cc:377] Logged in from keytab as oryx/127.27.225.193@KRBTEST.COM (short username oryx)
I20260806 21:06:52.833670 2403 webserver.cc:460] Webserver started at http://127.27.225.193:42619/ using document root <none> and password file <none>
I20260806 21:06:52.834306 2403 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:52.834400 2403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:52.834641 2403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:52.836740 2403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "513daa018ebf4514894d5ef4d7cb795e"
format_stamp: "Formatted at 2026-08-06 21:06:52 on dist-test-slave-69rg"
server_key: "bfd194a904feb9857712eb2fcf30e764"
server_key_iv: "b85d8eac4c13ab6dcd7863ca394397cd"
server_key_version: "encryptionkey@0"
I20260806 21:06:52.837327 2403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "513daa018ebf4514894d5ef4d7cb795e"
format_stamp: "Formatted at 2026-08-06 21:06:52 on dist-test-slave-69rg"
server_key: "bfd194a904feb9857712eb2fcf30e764"
server_key_iv: "b85d8eac4c13ab6dcd7863ca394397cd"
server_key_version: "encryptionkey@0"
I20260806 21:06:52.841096 2403 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 21:06:52.843662 2418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:52.844913 2403 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:06:52.845057 2403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "513daa018ebf4514894d5ef4d7cb795e"
format_stamp: "Formatted at 2026-08-06 21:06:52 on dist-test-slave-69rg"
server_key: "bfd194a904feb9857712eb2fcf30e764"
server_key_iv: "b85d8eac4c13ab6dcd7863ca394397cd"
server_key_version: "encryptionkey@0"
I20260806 21:06:52.845227 2403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:52.868696 2403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:52.872267 2403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:52.872488 2403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:52.873111 2403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:52.873989 2403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:52.874065 2403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:52.874137 2403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:52.874179 2403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:52.883628 2403 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:43373
I20260806 21:06:52.883723 2533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:43373 every 8 connection(s)
I20260806 21:06:52.884646 2403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:06:52.885152 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 2403
I20260806 21:06:52.885303 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:06:52.885691 28551 external_mini_cluster.cc:1428] Setting key 95fbbe832ed493af5d38c105e51acd4e
Aug 06 21:06:52 dist-test-slave-69rg krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050412, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.27.225.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.225.194@KRBTEST.COM" created.
I20260806 21:06:52.900279 2380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:52.886860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:39769 (local address 127.27.225.254:45179)
0806 21:06:52.887008 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:52.887014 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:52.888129 (+ 1115us) server_negotiation.cc:407] Connection header received
0806 21:06:52.889600 (+ 1471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:52.889603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:52.889662 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:52.889752 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:52.891014 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:52.891567 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:52.892749 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:52.892995 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:52.893111 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:52.895985 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:52.895988 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:52.895990 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:52.896018 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:52.897651 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:52.898294 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:52.898297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:52.898298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:52.898378 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:52.898857 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:52.898860 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:52.898862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:52.899147 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:52.899272 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:52.899995 (+ 723us) server_negotiation.cc:299] Negotiation successful
0806 21:06:52.900116 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 21:06:52.901572 2536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:52.887424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.193:39769)
0806 21:06:52.887979 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:52.888025 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:52.889386 (+ 1361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:52.889968 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:52.889976 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:52.890355 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:52.890859 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:52.890874 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:52.891716 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:52.891718 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:52.892606 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:52.892615 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:52.893167 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:52.893171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:52.893344 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:52.894041 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:52.894057 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:52.895799 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:52.897798 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:52.897805 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:52.897844 (+ 39us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:52.898152 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:52.898549 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:52.898552 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:52.898555 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:52.898745 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:52.899328 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:52.899337 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:52.899691 (+ 354us) client_negotiation.cc:769] Sending connection context
0806 21:06:52.899932 (+ 241us) client_negotiation.cc:240] Negotiation successful
0806 21:06:52.900346 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":164,"threads_started":1}
I20260806 21:06:52.903250 2534 heartbeater.cc:347] Connected to a master server at 127.27.225.254:45179
I20260806 21:06:52.903518 2534 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:52.904140 2534 heartbeater.cc:511] Master 127.27.225.254:45179 requested a full tablet report, sending...
I20260806 21:06:52.905983 2339 ts_manager.cc:194] Registered new tserver with Master: 513daa018ebf4514894d5ef4d7cb795e (127.27.225.193:43373)
I20260806 21:06:52.907130 2339 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.225.193@KRBTEST.COM'} at 127.27.225.193:39769
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.225.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.194.keytab.
Entry for principal oryx/127.27.225.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.194.keytab.
I20260806 21:06:52.918105 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:45179
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.194.keytab
--principal=oryx/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:46765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:53.035430 2539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:53.035732 2539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:53.035816 2539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:53.040057 2539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:53.040163 2539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:53.040252 2539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:06:53.045894 2539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:46765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.194.keytab
--principal=oryx/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:45179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:53.047024 2539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:53.048063 2539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:53.058135 2547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:53.058149 2544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:53.058149 2545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:53.058339 2539 server_base.cc:1034] running on GCE node
I20260806 21:06:53.058938 2539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:53.059672 2539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:53.060884 2539 hybrid_clock.cc:648] HybridClock initialized: now 1786050413060849 us; error 52 us; skew 500 ppm
Aug 06 21:06:53 dist-test-slave-69rg krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050413, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:53.063994 2539 init.cc:377] Logged in from keytab as oryx/127.27.225.194@KRBTEST.COM (short username oryx)
I20260806 21:06:53.064989 2539 webserver.cc:460] Webserver started at http://127.27.225.194:44841/ using document root <none> and password file <none>
I20260806 21:06:53.065572 2539 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:53.065676 2539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:53.065999 2539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:53.068136 2539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "130105c8e4b94acea85e45c8cbbfd8ad"
format_stamp: "Formatted at 2026-08-06 21:06:53 on dist-test-slave-69rg"
server_key: "1a8f859815d4822c99f003b5e054adb2"
server_key_iv: "b3f41df1f43f852bd75266ca3eeecf51"
server_key_version: "encryptionkey@0"
I20260806 21:06:53.068723 2539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "130105c8e4b94acea85e45c8cbbfd8ad"
format_stamp: "Formatted at 2026-08-06 21:06:53 on dist-test-slave-69rg"
server_key: "1a8f859815d4822c99f003b5e054adb2"
server_key_iv: "b3f41df1f43f852bd75266ca3eeecf51"
server_key_version: "encryptionkey@0"
I20260806 21:06:53.072518 2539 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:06:53.075256 2554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:53.076637 2539 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 21:06:53.076762 2539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "130105c8e4b94acea85e45c8cbbfd8ad"
format_stamp: "Formatted at 2026-08-06 21:06:53 on dist-test-slave-69rg"
server_key: "1a8f859815d4822c99f003b5e054adb2"
server_key_iv: "b3f41df1f43f852bd75266ca3eeecf51"
server_key_version: "encryptionkey@0"
I20260806 21:06:53.076867 2539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:53.095556 2539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:53.098737 2539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:53.098902 2539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:53.099465 2539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:53.100271 2539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:53.100374 2539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:53.100435 2539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:53.100459 2539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:53.109396 2539 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:42173
I20260806 21:06:53.109421 2669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:42173 every 8 connection(s)
I20260806 21:06:53.110333 2539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:06:53.115880 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 2539
I20260806 21:06:53.115980 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:06:53.116269 28551 external_mini_cluster.cc:1428] Setting key 30a5afb23ffea806b3da299fca7e8798
Aug 06 21:06:53 dist-test-slave-69rg krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050413, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM
I20260806 21:06:53.127189 2380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:53.112468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:44705 (local address 127.27.225.254:45179)
0806 21:06:53.112606 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:53.112610 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:53.113652 (+ 1042us) server_negotiation.cc:407] Connection header received
0806 21:06:53.115141 (+ 1489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:53.115145 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:53.115251 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:53.115351 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:53.116708 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.117361 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.119126 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.119365 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.119452 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:53.122600 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:53.122608 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:53.122614 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:53.122657 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:53.124439 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:53.125274 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:53.125277 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:53.125278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:53.125375 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:53.125899 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:53.125901 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:53.125902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:53.126103 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:53.126182 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:53.126929 (+ 747us) server_negotiation.cc:299] Negotiation successful
0806 21:06:53.127051 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
WARNING: no policy specified for oryx/127.27.225.195@KRBTEST.COM; defaulting to no policy
I20260806 21:06:53.128445 2672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:53.112911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.194:44705)
0806 21:06:53.113496 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:53.113539 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:53.114925 (+ 1386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:53.115536 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:53.115542 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:53.115968 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:53.116547 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:53.116564 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.117504 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.117508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:53.118961 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:53.118977 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.119538 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.119545 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:53.119802 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:53.120631 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:53.120649 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:53.122354 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:53.124613 (+ 2259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:53.124625 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:53.124659 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:53.125142 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:53.125480 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:53.125486 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:53.125490 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:53.125720 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:53.126225 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:53.126235 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:53.126620 (+ 385us) client_negotiation.cc:769] Sending connection context
0806 21:06:53.126875 (+ 255us) client_negotiation.cc:240] Negotiation successful
0806 21:06:53.127281 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":109,"threads_started":1}
I20260806 21:06:53.129642 2670 heartbeater.cc:347] Connected to a master server at 127.27.225.254:45179
I20260806 21:06:53.129937 2670 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:53.130451 2670 heartbeater.cc:511] Master 127.27.225.254:45179 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.225.195@KRBTEST.COM" created.
I20260806 21:06:53.131623 2339 ts_manager.cc:194] Registered new tserver with Master: 130105c8e4b94acea85e45c8cbbfd8ad (127.27.225.194:42173)
I20260806 21:06:53.132331 2339 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.225.194@KRBTEST.COM'} at 127.27.225.194:44705
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.225.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.195.keytab.
Entry for principal oryx/127.27.225.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.195.keytab.
I20260806 21:06:53.149538 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:45179
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.195.keytab
--principal=oryx/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:46765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:53.267768 2675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:53.268071 2675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:53.268118 2675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:53.272347 2675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:53.272428 2675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:53.272492 2675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:06:53.278283 2675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:46765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.195.keytab
--principal=oryx/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:45179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:53.279436 2675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:53.280653 2675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:06:53.291428 2675 server_base.cc:1034] running on GCE node
W20260806 21:06:53.291393 2680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:53.291359 2681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:53.291370 2683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:53.292143 2675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:53.292819 2675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:53.294042 2675 hybrid_clock.cc:648] HybridClock initialized: now 1786050413294026 us; error 32 us; skew 500 ppm
Aug 06 21:06:53 dist-test-slave-69rg krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050413, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:53.298882 2675 init.cc:377] Logged in from keytab as oryx/127.27.225.195@KRBTEST.COM (short username oryx)
I20260806 21:06:53.300103 2675 webserver.cc:460] Webserver started at http://127.27.225.195:36289/ using document root <none> and password file <none>
I20260806 21:06:53.300797 2675 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:53.300880 2675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:53.301074 2675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:53.303493 2675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "3ae6436ae2ee446eacad8af409ce0daf"
format_stamp: "Formatted at 2026-08-06 21:06:53 on dist-test-slave-69rg"
server_key: "79083649b13da6e93800e1fb3b6c971a"
server_key_iv: "2c2a265030cd44c8ad601ef3cb4f096d"
server_key_version: "encryptionkey@0"
I20260806 21:06:53.304096 2675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "3ae6436ae2ee446eacad8af409ce0daf"
format_stamp: "Formatted at 2026-08-06 21:06:53 on dist-test-slave-69rg"
server_key: "79083649b13da6e93800e1fb3b6c971a"
server_key_iv: "2c2a265030cd44c8ad601ef3cb4f096d"
server_key_version: "encryptionkey@0"
I20260806 21:06:53.308324 2675 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260806 21:06:53.311534 2690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:53.313320 2675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 21:06:53.313490 2675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "3ae6436ae2ee446eacad8af409ce0daf"
format_stamp: "Formatted at 2026-08-06 21:06:53 on dist-test-slave-69rg"
server_key: "79083649b13da6e93800e1fb3b6c971a"
server_key_iv: "2c2a265030cd44c8ad601ef3cb4f096d"
server_key_version: "encryptionkey@0"
I20260806 21:06:53.313654 2675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:53.326766 2675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:53.330647 2675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:53.330878 2675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:53.331580 2675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:53.332490 2675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:53.332541 2675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:53.332631 2675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:53.332799 2675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:53.343765 2675 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:43009
I20260806 21:06:53.343847 2804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:43009 every 8 connection(s)
I20260806 21:06:53.344830 2675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:06:53.347980 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 2675
I20260806 21:06:53.348136 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:06:53.348518 28551 external_mini_cluster.cc:1428] Setting key 53221c639b178cc3122acbd11146bd30
Aug 06 21:06:53 dist-test-slave-69rg krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050413, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM
I20260806 21:06:53.360904 2380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:53.347019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:44825 (local address 127.27.225.254:45179)
0806 21:06:53.347218 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:53.347222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:53.348312 (+ 1090us) server_negotiation.cc:407] Connection header received
0806 21:06:53.350389 (+ 2077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:53.350392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:53.350449 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:53.350529 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:53.351907 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.352432 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.353670 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.354010 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.354127 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:53.357009 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:53.357012 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:53.357015 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:53.357042 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:53.358757 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:53.359354 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:53.359357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:53.359359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:53.359447 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:53.359799 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:53.359801 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:53.359802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:53.359995 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:53.360075 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:53.360653 (+ 578us) server_negotiation.cc:299] Negotiation successful
0806 21:06:53.360773 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 21:06:53.362087 2807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:53.347432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.195:44825)
0806 21:06:53.348012 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:53.348090 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:53.350109 (+ 2019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:53.350706 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:53.350714 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:53.351250 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:53.351773 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:53.351787 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.352544 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.352546 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:53.353533 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:53.353543 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.354293 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.354297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:53.354473 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:53.355132 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:53.355151 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:53.356835 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:53.358894 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:53.358899 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:53.358912 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:53.359229 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:53.359549 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:53.359551 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:53.359553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:53.359714 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:53.360117 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:53.360123 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:53.360420 (+ 297us) client_negotiation.cc:769] Sending connection context
0806 21:06:53.360602 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 21:06:53.360905 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":206,"threads_started":1}
I20260806 21:06:53.363358 2805 heartbeater.cc:347] Connected to a master server at 127.27.225.254:45179
I20260806 21:06:53.363628 2805 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:53.364156 2805 heartbeater.cc:511] Master 127.27.225.254:45179 requested a full tablet report, sending...
I20260806 21:06:53.365281 2338 ts_manager.cc:194] Registered new tserver with Master: 3ae6436ae2ee446eacad8af409ce0daf (127.27.225.195:43009)
I20260806 21:06:53.365950 2338 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.225.195@KRBTEST.COM'} at 127.27.225.195:44825
I20260806 21:06:53.376051 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.27.225.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.225.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.225.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.253.keytab.
Entry for principal oryx/127.27.225.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.253.keytab.
I20260806 21:06:53.410676 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.253:34129
--webserver_interface=127.27.225.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.253.keytab
--principal=oryx/127.27.225.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:46765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:45179,127.27.225.253:34129
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:06:53.525274 2810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:53.525571 2810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:53.525620 2810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:53.529736 2810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:06:53.529831 2810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:53.529853 2810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:53.529870 2810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:06:53.529937 2810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:06:53.535804 2810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:46765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:45179,127.27.225.253:34129
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.253.keytab
--principal=oryx/127.27.225.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.253:34129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.225.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
W20260806 21:06:53.536068 2810 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.27.225.254:45179,127.27.225.253:34129'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 21:06:53.537000 2810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:53.538126 2810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:53.547107 2815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:53.547130 2818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:53.547123 2816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:53.547264 2810 server_base.cc:1034] running on GCE node
I20260806 21:06:53.548063 2810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:53.549350 2810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:53.550554 2810 hybrid_clock.cc:648] HybridClock initialized: now 1786050413550530 us; error 53 us; skew 500 ppm
Aug 06 21:06:53 dist-test-slave-69rg krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050413, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:53.553735 2810 init.cc:377] Logged in from keytab as oryx/127.27.225.253@KRBTEST.COM (short username oryx)
I20260806 21:06:53.554901 2810 webserver.cc:460] Webserver started at http://127.27.225.253:37443/ using document root <none> and password file <none>
I20260806 21:06:53.555598 2810 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:53.555708 2810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:53.555984 2810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:53.558471 2810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/data/instance:
uuid: "4859fba6ee99437aa8a0285c7270fe12"
format_stamp: "Formatted at 2026-08-06 21:06:53 on dist-test-slave-69rg"
server_key: "57966fce3e6613343575cea0b20e1f52"
server_key_iv: "2dbc72e54646626420895ad1a2b27217"
server_key_version: "encryptionkey@0"
I20260806 21:06:53.559116 2810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/wal/instance:
uuid: "4859fba6ee99437aa8a0285c7270fe12"
format_stamp: "Formatted at 2026-08-06 21:06:53 on dist-test-slave-69rg"
server_key: "57966fce3e6613343575cea0b20e1f52"
server_key_iv: "2dbc72e54646626420895ad1a2b27217"
server_key_version: "encryptionkey@0"
I20260806 21:06:53.563288 2810 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:06:53.565955 2825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:53.567297 2810 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:06:53.567485 2810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/wal
uuid: "4859fba6ee99437aa8a0285c7270fe12"
format_stamp: "Formatted at 2026-08-06 21:06:53 on dist-test-slave-69rg"
server_key: "57966fce3e6613343575cea0b20e1f52"
server_key_iv: "2dbc72e54646626420895ad1a2b27217"
server_key_version: "encryptionkey@0"
I20260806 21:06:53.567619 2810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:53.588624 2810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:53.592300 2810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:53.592561 2810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:53.601315 2810 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.253:34129
I20260806 21:06:53.601632 2877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.253:34129 every 8 connection(s)
I20260806 21:06:53.602492 2810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/data/info.pb
I20260806 21:06:53.605455 2878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:53.609504 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 2810
I20260806 21:06:53.609407 2878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 21:06:53 dist-test-slave-69rg krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050413, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.253@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM
Aug 06 21:06:53 dist-test-slave-69rg krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.225.253@KRBTEST.COM
I20260806 21:06:53.623814 2380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:53.611551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52552 (local address 127.27.225.254:45179)
0806 21:06:53.611692 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:53.611698 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:53.612278 (+ 580us) server_negotiation.cc:407] Connection header received
0806 21:06:53.613121 (+ 843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:53.613125 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:53.613195 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:53.613310 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:53.614318 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.614908 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.616041 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.616309 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.616434 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:53.619592 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:53.619595 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:53.619597 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:53.619629 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:53.621345 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:53.622053 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:53.622056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:53.622058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:53.622188 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:53.622617 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:53.622621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:53.622623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:53.622887 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:53.623016 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:53.623478 (+ 462us) server_negotiation.cc:299] Negotiation successful
0806 21:06:53.623618 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 21:06:53.624725 2882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:53.611451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49262 (local address 127.27.225.253:34129)
0806 21:06:53.611982 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:53.612000 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:53.612042 (+ 42us) server_negotiation.cc:407] Connection header received
0806 21:06:53.612883 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:53.612904 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:53.613103 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:53.613300 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:53.614284 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.615175 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.616101 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.616466 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.616638 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:53.620005 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:53.620011 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:53.620021 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:53.620042 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:53.621995 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:53.622450 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:53.622463 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:53.622464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:53.622540 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:53.622834 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:53.622838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:53.622840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:53.623219 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:53.623444 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:53.623631 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 21:06:53.623861 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":366,"spinlock_wait_cycles":61184,"thread_start_us":212,"threads_started":1}
I20260806 21:06:53.624722 2881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:53.611465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.0.0.1:52552)
0806 21:06:53.612051 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:53.612097 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:53.612903 (+ 806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:53.613379 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:53.613385 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:53.613802 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:53.614160 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:53.614175 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.615057 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.615065 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:53.615901 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:53.615910 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.616493 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.616498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:53.616638 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:53.617481 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:53.617498 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:53.619400 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:53.621478 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:53.621488 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:53.621498 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:53.621923 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:53.622324 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:53.622330 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:53.622332 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:53.622499 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:53.623011 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:53.623019 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:53.623229 (+ 210us) client_negotiation.cc:769] Sending connection context
0806 21:06:53.623411 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 21:06:53.623786 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"spinlock_wait_cycles":1792,"thread_start_us":160,"threads_started":1}
I20260806 21:06:53.626346 2878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 21:06:53 dist-test-slave-69rg krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050413, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.253@KRBTEST.COM for oryx/127.27.225.253@KRBTEST.COM
I20260806 21:06:53.635293 2881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:53.626900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:34129 (local address 127.0.0.1:49266)
0806 21:06:53.627009 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:53.627029 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:53.627106 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:53.627379 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:53.627382 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:53.627550 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:53.627656 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:53.627661 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.628441 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.628444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:53.629007 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:53.629015 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.630167 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.630170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:53.630249 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:53.630812 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:53.630825 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:53.632189 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:53.633928 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:53.633931 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:53.633933 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:53.634172 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:53.634428 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:53.634430 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:53.634431 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:53.634512 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:53.634880 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:53.634882 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:53.634962 (+ 80us) client_negotiation.cc:769] Sending connection context
0806 21:06:53.635049 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 21:06:53.635168 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 21:06:53.635389 2882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:53.626972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49266 (local address 127.27.225.253:34129)
0806 21:06:53.627091 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:53.627096 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:53.627112 (+ 16us) server_negotiation.cc:407] Connection header received
0806 21:06:53.627213 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:53.627216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:53.627268 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:53.627329 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:53.627801 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.628331 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.629100 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.630056 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.630144 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:53.632323 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:53.632327 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:53.632329 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:53.632349 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:53.633750 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:53.634262 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:53.634265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:53.634266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:53.634338 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:53.634590 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:53.634592 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:53.634593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:53.634797 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:53.634872 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:53.635150 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 21:06:53.635251 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 21:06:53.638464 2878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12: Bootstrap starting.
I20260806 21:06:53.640741 2878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:53.641469 2878 log.cc:824] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:53.643500 2878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12: No bootstrap required, opened a new log
I20260806 21:06:53.646523 2878 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } }
I20260806 21:06:53.646720 2878 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:53.646754 2878 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4859fba6ee99437aa8a0285c7270fe12, State: Initialized, Role: FOLLOWER
I20260806 21:06:53.647106 2878 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } }
I20260806 21:06:53.647528 2884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } } }
I20260806 21:06:53.647652 2884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:06:53.648257 2878 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 21:06:53.653406 2895 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:06:53.653522 2895 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:06:53.656294 2810 master_runner.cc:428] Detected that this master 4859fba6ee99437aa8a0285c7270fe12 is joining an existing cluster
I20260806 21:06:53.656358 2810 master_runner.cc:432] Initiating AddMaster RPC to add 127.27.225.253:34129
I20260806 21:06:53.667476 2901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:53.658760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:34129 (local address 127.0.0.1:49278)
0806 21:06:53.659124 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:53.659145 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:53.659245 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:53.659542 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:53.659545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:53.659798 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:53.659937 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:53.659944 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.660983 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.660986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:53.661978 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:53.661988 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.662593 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.662597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:53.662693 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:53.663251 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:53.663267 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:53.663707 (+ 440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:53.665796 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:53.665799 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:53.665801 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:53.666045 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:53.666345 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:53.666351 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:53.666353 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:53.666444 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:53.666881 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:53.666884 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:53.666980 (+ 96us) client_negotiation.cc:769] Sending connection context
0806 21:06:53.667081 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 21:06:53.667229 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":60,"thread_start_us":148,"threads_started":1}
I20260806 21:06:53.667519 2882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:53.659253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49278 (local address 127.27.225.253:34129)
0806 21:06:53.659360 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:53.659366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:53.659382 (+ 16us) server_negotiation.cc:407] Connection header received
0806 21:06:53.659410 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:53.659412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:53.659442 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:53.659531 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:53.660061 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.660850 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.662106 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.662463 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.662575 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:53.663851 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:53.663859 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:53.663865 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:53.663894 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:53.665639 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:53.666148 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:53.666154 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:53.666159 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:53.666245 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:53.666541 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:53.666547 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:53.666549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:53.666760 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:53.666904 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:53.667169 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 21:06:53.667301 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 21:06:53.667814 2380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:53.658750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52564 (local address 127.27.225.254:45179)
0806 21:06:53.658936 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:53.658944 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:53.659680 (+ 736us) server_negotiation.cc:407] Connection header received
0806 21:06:53.659845 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:53.659848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:53.659906 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:53.659995 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:53.660480 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.661079 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.662079 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.662467 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.662574 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:53.663962 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:53.663967 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:53.663969 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:53.663997 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:53.665919 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:53.666424 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:53.666428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:53.666430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:53.666519 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:53.666830 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:53.666837 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:53.666842 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:53.667100 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:53.667218 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:53.667572 (+ 354us) server_negotiation.cc:299] Negotiation successful
0806 21:06:53.667696 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 21:06:53.668458 2902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:53.659262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.0.0.1:52564)
0806 21:06:53.659575 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:53.659588 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:53.659721 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:53.660025 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:53.660028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:53.660253 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:53.660353 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:53.660359 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.661202 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.661209 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:53.661944 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:53.661953 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.662717 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.662720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:53.662800 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:53.663428 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:53.663452 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:53.663820 (+ 368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:53.666042 (+ 2222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:53.666045 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:53.666047 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:53.666295 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:53.666632 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:53.666635 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:53.666637 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:53.666720 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:53.667268 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:53.667272 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:53.667364 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 21:06:53.668103 (+ 739us) client_negotiation.cc:240] Negotiation successful
0806 21:06:53.668264 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":1,"thread_start_us":129,"threads_started":1}
Aug 06 21:06:53 dist-test-slave-69rg krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050412, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.254@KRBTEST.COM for oryx/127.27.225.253@KRBTEST.COM
I20260806 21:06:53.684109 2882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:53.673636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49290 (local address 127.27.225.253:34129)
0806 21:06:53.673787 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:53.673792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:53.674217 (+ 425us) server_negotiation.cc:407] Connection header received
0806 21:06:53.674403 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:53.674405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:53.674452 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:53.674523 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:53.675305 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.675855 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.677097 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.677335 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.677459 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:53.680231 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:53.680239 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:53.680246 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:53.680281 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:53.682150 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:53.682696 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:53.682701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:53.682703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:53.682790 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:53.683111 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:53.683117 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:53.683118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:53.683314 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:53.683433 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:53.683803 (+ 370us) server_negotiation.cc:299] Negotiation successful
0806 21:06:53.683932 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 21:06:53.684170 2903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:53.673605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:34129 (local address 127.0.0.1:49290)
0806 21:06:53.674060 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:53.674128 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:53.674262 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:53.674580 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:53.674586 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:53.675039 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:53.675184 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:53.675192 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.675964 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.675967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:53.676978 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:53.676990 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.677449 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.677453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:53.677569 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:53.678153 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:53.678169 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:53.680026 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:53.682281 (+ 2255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:53.682292 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:53.682300 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:53.682574 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:53.682903 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:53.682907 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:53.682909 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:53.683010 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:53.683435 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:53.683445 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:53.683584 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 21:06:53.683745 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 21:06:53.683967 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":131,"threads_started":1}
I20260806 21:06:53.685879 2338 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.27.225.253:34129: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12"
member_type: NON_VOTER
last_known_addr {
host: "127.27.225.253"
port: 34129
}
attrs {
promote: true
}
}
dest_uuid: "00de4fec8340498da672d706bcc37acf"
cas_config_opid_index: -1
I20260806 21:06:53.686651 2338 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: NON_VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: true } }
I20260806 21:06:53.688759 2906 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 4859fba6ee99437aa8a0285c7270fe12 (127.27.225.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: NON_VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: true } } }
I20260806 21:06:53.689430 2852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:53.690469 2906 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.27.225.253:34129
I20260806 21:06:53.690810 2852 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 1 FOLLOWER]: Refusing update from remote peer 00de4fec8340498da672d706bcc37acf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 21:06:53.690793 2906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "00de4fec8340498da672d706bcc37acf" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: NON_VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: true } } }
I20260806 21:06:53.690883 2906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:53.691606 2810 master.cc:537] Master@127.27.225.253:34129 shutting down...
I20260806 21:06:53.691576 2907 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: NON_VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 21:06:53.694199 2323 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 21:06:53.695559 2323 consensus_peers.cc:487] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf -> Peer 4859fba6ee99437aa8a0285c7270fe12 (127.27.225.253:34129): Couldn't send request to peer 4859fba6ee99437aa8a0285c7270fe12. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 21:06:53.696475 2810 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:06:53.696799 2810 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:53.696923 2810 tablet_replica.cc:330] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12: stopping tablet replica
WARNING: no policy specified for oryx/127.27.225.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.225.252@KRBTEST.COM" created.
I20260806 21:06:53.712720 2810 master.cc:552] Master@127.27.225.253:34129 shutdown complete.
I20260806 21:06:53.712797 2810 master_runner.cc:305] Clearing existing system tablet
I20260806 21:06:53.713058 2810 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:53.713119 2810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:53.715085 2810 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:06:53.716289 2912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:53.716670 2810 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:53.716825 2810 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 21:06:53.716861 2810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/wal
uuid: "4859fba6ee99437aa8a0285c7270fe12"
format_stamp: "Formatted at 2026-08-06 21:06:53 on dist-test-slave-69rg"
server_key: "57966fce3e6613343575cea0b20e1f52"
server_key_iv: "2dbc72e54646626420895ad1a2b27217"
server_key_version: "encryptionkey@0"
I20260806 21:06:53.719875 2810 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12: Deleting tablet data with delete state TABLET_DATA_DELETED
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.225.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.252.keytab.
Entry for principal oryx/127.27.225.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.252.keytab.
I20260806 21:06:53.721611 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.252:34577
--webserver_interface=127.27.225.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.252.keytab
--principal=oryx/127.27.225.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:46765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:45179,127.27.225.253:34129,127.27.225.252:34577
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:06:53.722972 2810 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 21:06:53.723083 2810 log.cc:1196] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 21:06:53.723428 2810 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12: Deleting consensus metadata
I20260806 21:06:53.723611 2810 master_runner.cc:315] Copying system tablet from 127.27.225.254:45179
I20260806 21:06:53.727335 2810 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12: tablet copy: Beginning tablet copy session from remote peer at address 127.27.225.254:45179
I20260806 21:06:53.744459 2919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:53.734280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.0.0.1:52568)
0806 21:06:53.734613 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:53.734633 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:53.734862 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:53.735404 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:53.735410 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:53.735718 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:53.735933 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:53.735945 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.737327 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.737334 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:53.738567 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:53.738584 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.739525 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.739528 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:53.739612 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:53.740389 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:53.740402 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:53.740778 (+ 376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:53.742774 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:53.742778 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:53.742780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:53.743023 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:53.743359 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:53.743361 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:53.743362 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:53.743433 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:53.743840 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:53.743843 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:53.743948 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 21:06:53.744068 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 21:06:53.744235 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":111,"threads_started":1}
I20260806 21:06:53.744450 2380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:53.734382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52568 (local address 127.27.225.254:45179)
0806 21:06:53.734537 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:53.734544 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:53.734752 (+ 208us) server_negotiation.cc:407] Connection header received
0806 21:06:53.735017 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:53.735024 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:53.735094 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:53.735171 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:53.736121 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.737112 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.738767 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.739237 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.739338 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:53.740954 (+ 1616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:53.740958 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:53.740961 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:53.740999 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:53.742600 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:53.743145 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:53.743149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:53.743151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:53.743246 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:53.743530 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:53.743534 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:53.743535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:53.743737 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:53.743851 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:53.744138 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 21:06:53.744272 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 21:06:53.745489 2361 tablet_copy_service.cc:140] P 00de4fec8340498da672d706bcc37acf: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 4859fba6ee99437aa8a0285c7270fe12 ({username='oryx', principal='oryx/127.27.225.253@KRBTEST.COM'} at 127.0.0.1:52568)
I20260806 21:06:53.745695 2361 tablet_copy_service.cc:161] P 00de4fec8340498da672d706bcc37acf: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 4859fba6ee99437aa8a0285c7270fe12 at {username='oryx', principal='oryx/127.27.225.253@KRBTEST.COM'} at 127.0.0.1:52568: session id = 4859fba6ee99437aa8a0285c7270fe12-00000000000000000000000000000000
I20260806 21:06:53.747283 2361 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 21:06:53.749135 2810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 21:06:53.751453 2810 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12: tablet copy: Starting download of 0 data blocks...
I20260806 21:06:53.751710 2810 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12: tablet copy: Starting download of 1 WAL segments...
I20260806 21:06:53.753566 2810 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 21:06:53.755940 2361 tablet_copy_service.cc:342] P 00de4fec8340498da672d706bcc37acf: Request end of tablet copy session 4859fba6ee99437aa8a0285c7270fe12-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.27.225.253@KRBTEST.COM'} at 127.0.0.1:52568
I20260806 21:06:53.756156 2361 tablet_copy_service.cc:433] P 00de4fec8340498da672d706bcc37acf: ending tablet copy session 4859fba6ee99437aa8a0285c7270fe12-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 4859fba6ee99437aa8a0285c7270fe12
I20260806 21:06:53.763794 2810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:53.765609 2920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:53.765933 2923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:53.766134 2921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:53.766587 2810 server_base.cc:1034] running on GCE node
I20260806 21:06:53.766789 2810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:53.767122 2810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:53.768286 2810 hybrid_clock.cc:648] HybridClock initialized: now 1786050413768275 us; error 28 us; skew 500 ppm
Aug 06 21:06:53 dist-test-slave-69rg krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050413, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:53.769964 2810 init.cc:377] Logged in from keytab as oryx/127.27.225.253@KRBTEST.COM (short username oryx)
I20260806 21:06:53.770519 2810 webserver.cc:460] Webserver started at http://127.27.225.253:42161/ using document root <none> and password file <none>
I20260806 21:06:53.770740 2810 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:53.770802 2810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:53.772122 2810 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 21:06:53.773169 2930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:53.773432 2810 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 21:06:53.773535 2810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/wal
uuid: "4859fba6ee99437aa8a0285c7270fe12"
format_stamp: "Formatted at 2026-08-06 21:06:53 on dist-test-slave-69rg"
server_key: "57966fce3e6613343575cea0b20e1f52"
server_key_iv: "2dbc72e54646626420895ad1a2b27217"
server_key_version: "encryptionkey@0"
I20260806 21:06:53.773653 2810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:53.791752 2810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:53.794164 2810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:53.799764 2810 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.253:34129
I20260806 21:06:53.799779 2982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.253:34129 every 8 connection(s)
I20260806 21:06:53.800482 2810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-1/data/info.pb
I20260806 21:06:53.802227 2983 sys_catalog.cc:263] Verifying existing consensus state
I20260806 21:06:53.803186 2983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12: Bootstrap starting.
I20260806 21:06:53.812963 2983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 21:06:53.813323 2983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12: Bootstrap complete.
I20260806 21:06:53.813947 2983 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: NON_VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: true } }
I20260806 21:06:53.814085 2983 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4859fba6ee99437aa8a0285c7270fe12, State: Initialized, Role: LEARNER
I20260806 21:06:53.814206 2983 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: NON_VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: true } }
I20260806 21:06:53.814607 2986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: NON_VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: true } } }
I20260806 21:06:53.814791 2983 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:06:53.814800 2986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [sys.catalog]: This master's current role is: LEARNER
I20260806 21:06:53.818598 2998 catalog_manager.cc:1259] Loaded cluster ID: 42ffcff42f5a4ac2a89ff0d847fddd0b
I20260806 21:06:53.818671 2998 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12: loading cluster ID for follower catalog manager: success
I20260806 21:06:53.820384 2998 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12: acquiring CA information for follower catalog manager: success
I20260806 21:06:53.821559 2998 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 21:06:53.886417 2913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:53.886718 2913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:53.886766 2913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:53.891420 2913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:06:53.891484 2913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:53.891503 2913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:53.891521 2913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:06:53.891537 2913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:06:53.897042 2913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:46765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:45179,127.27.225.253:34129,127.27.225.252:34577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.252.keytab
--principal=oryx/127.27.225.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.252:34577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.225.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:53.898368 2913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:53.899511 2913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:53.908576 3003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:53.908614 3004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:53.908622 3006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:53.909137 2913 server_base.cc:1034] running on GCE node
I20260806 21:06:53.909441 2534 heartbeater.cc:503] Master 127.27.225.254:45179 was elected leader, sending a full tablet report...
I20260806 21:06:53.909909 2913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:53.911461 2913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:53.912662 2913 hybrid_clock.cc:648] HybridClock initialized: now 1786050413912604 us; error 65 us; skew 500 ppm
Aug 06 21:06:53 dist-test-slave-69rg krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050413, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:53.916007 2913 init.cc:377] Logged in from keytab as oryx/127.27.225.252@KRBTEST.COM (short username oryx)
I20260806 21:06:53.917054 2913 webserver.cc:460] Webserver started at http://127.27.225.252:37825/ using document root <none> and password file <none>
I20260806 21:06:53.917723 2913 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:53.917821 2913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:53.918088 2913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:06:53.920327 2913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/data/instance:
uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131"
format_stamp: "Formatted at 2026-08-06 21:06:53 on dist-test-slave-69rg"
server_key: "e8622e5bc5010d59d184df2aeaa258f9"
server_key_iv: "7263ef1667ad1aff07e1c1dda03b1340"
server_key_version: "encryptionkey@0"
I20260806 21:06:53.920989 2913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/wal/instance:
uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131"
format_stamp: "Formatted at 2026-08-06 21:06:53 on dist-test-slave-69rg"
server_key: "e8622e5bc5010d59d184df2aeaa258f9"
server_key_iv: "7263ef1667ad1aff07e1c1dda03b1340"
server_key_version: "encryptionkey@0"
I20260806 21:06:53.924929 2913 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:06:53.927533 3015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:53.928876 2913 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 21:06:53.929035 2913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/wal
uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131"
format_stamp: "Formatted at 2026-08-06 21:06:53 on dist-test-slave-69rg"
server_key: "e8622e5bc5010d59d184df2aeaa258f9"
server_key_iv: "7263ef1667ad1aff07e1c1dda03b1340"
server_key_version: "encryptionkey@0"
I20260806 21:06:53.929153 2913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:53.964005 2913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:53.968019 2913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:53.968277 2913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:53.977206 2913 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.252:34577
I20260806 21:06:53.977515 3067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.252:34577 every 8 connection(s)
I20260806 21:06:53.978415 2913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/data/info.pb
I20260806 21:06:53.981314 3068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:53.982439 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 2913
I20260806 21:06:53.987591 3068 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 21:06:53 dist-test-slave-69rg krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.225.252@KRBTEST.COM
Aug 06 21:06:53 dist-test-slave-69rg krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050413, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.252@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM
I20260806 21:06:53.998974 3070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:53.984168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55034 (local address 127.27.225.252:34577)
0806 21:06:53.984604 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:53.984619 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:53.984657 (+ 38us) server_negotiation.cc:407] Connection header received
0806 21:06:53.985302 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:53.985318 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:53.985656 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:53.986058 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:53.987042 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.988117 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.989287 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.990030 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.990182 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:53.993464 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:53.993470 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:53.993483 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:53.993515 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:53.995817 (+ 2302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:53.996366 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:53.996371 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:53.996376 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:53.996546 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:53.996878 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:53.996880 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:53.996882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:53.997287 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:53.997512 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:53.997876 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 21:06:53.998172 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":138,"threads_started":1}
I20260806 21:06:54.000735 2380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:53.989860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52572 (local address 127.27.225.254:45179)
0806 21:06:53.989996 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:53.990001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:53.990262 (+ 261us) server_negotiation.cc:407] Connection header received
0806 21:06:53.990543 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:53.990550 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:53.990640 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:53.990742 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:53.991769 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.992632 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.993918 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.994294 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.994392 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:53.997118 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:53.997122 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:53.997124 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:53.997159 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:53.998714 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:53.999244 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:53.999249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:53.999251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:53.999354 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:53.999636 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:53.999639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:53.999641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:53.999860 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:53.999962 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.000448 (+ 486us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.000618 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 21:06:54.000778 3071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:53.989631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.0.0.1:52572)
0806 21:06:53.990073 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:53.990179 (+ 106us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:53.990362 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:53.990853 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:53.990861 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:53.991399 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:53.991596 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:53.991608 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.992822 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.992825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:53.993733 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:53.993745 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:53.994429 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:53.994436 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:53.994571 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:53.995002 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:53.995021 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:53.996960 (+ 1939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:53.998829 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:53.998839 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:53.998845 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:53.999134 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:53.999464 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:53.999466 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:53.999468 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:53.999544 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:53.999992 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:53.999998 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:54.000115 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 21:06:54.000364 (+ 249us) client_negotiation.cc:240] Negotiation successful
0806 21:06:54.000565 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":131,"threads_started":1}
I20260806 21:06:54.002843 3068 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 21:06:54 dist-test-slave-69rg krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050413, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.252@KRBTEST.COM for oryx/127.27.225.253@KRBTEST.COM
I20260806 21:06:54.012295 3071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.003296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:34129 (local address 127.0.0.1:49294)
0806 21:06:54.003446 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:54.003459 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:54.003528 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:54.004006 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:54.004009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:54.004156 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:54.004249 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.004254 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.005198 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.005201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.005727 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.005734 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.006265 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.006268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.006341 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.006839 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:54.006850 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:54.008180 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:54.010453 (+ 2273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.010456 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.010457 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.010717 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.011060 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.011067 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.011072 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.011216 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.011730 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:54.011732 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:54.011827 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 21:06:54.011987 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 21:06:54.012139 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 21:06:54.012465 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.003485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49294 (local address 127.27.225.253:34129)
0806 21:06:54.003789 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.003793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.003805 (+ 12us) server_negotiation.cc:407] Connection header received
0806 21:06:54.003844 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.003846 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.003887 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.003965 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:54.004355 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.005083 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.005869 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.006130 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.006245 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.008364 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:54.008367 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:54.008370 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:54.008397 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:54.010301 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.010821 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.010824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.010825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.010914 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.011303 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.011306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.011307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.011530 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:54.011643 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.012123 (+ 480us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.012279 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":119,"threads_started":1}
I20260806 21:06:54.013803 3068 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 21:06:54 dist-test-slave-69rg krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050413, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.252@KRBTEST.COM for oryx/127.27.225.252@KRBTEST.COM
I20260806 21:06:54.022864 3071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.014511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:34577 (local address 127.0.0.1:55042)
0806 21:06:54.014663 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:54.014678 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:54.014763 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:54.014999 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:54.015002 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:54.015224 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:54.015296 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.015302 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.016672 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.016675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.017267 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.017277 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.017781 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.017784 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.017891 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.018393 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:54.018404 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:54.019735 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:54.021345 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.021348 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.021350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.021539 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.021854 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.021857 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.021858 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.021937 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.022320 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:54.022323 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:54.022410 (+ 87us) client_negotiation.cc:769] Sending connection context
0806 21:06:54.022512 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 21:06:54.022666 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 21:06:54.022879 3070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.014632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55042 (local address 127.27.225.252:34577)
0806 21:06:54.014740 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.014744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.014757 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:06:54.014841 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.014846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.014887 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.014989 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:54.015411 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.016082 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.017386 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.017625 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.017753 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.019880 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:54.019883 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:54.019885 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:54.019905 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:54.021224 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.021635 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.021640 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.021642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.021732 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.022027 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.022032 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.022034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.022225 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:54.022332 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.022567 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.022760 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":80,"server-negotiator.queue_time_us":50}
I20260806 21:06:54.026208 3068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131: Bootstrap starting.
I20260806 21:06:54.028692 3068 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:54.029477 3068 log.cc:824] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:54.031654 3068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131: No bootstrap required, opened a new log
I20260806 21:06:54.035352 3068 raft_consensus.cc:348] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } }
I20260806 21:06:54.035576 3068 raft_consensus.cc:374] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:54.035666 3068 raft_consensus.cc:729] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db766e7b4c4b4ae1ba460ea4cf5a6131, State: Initialized, Role: FOLLOWER
I20260806 21:06:54.036108 3068 consensus_queue.cc:260] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } }
I20260806 21:06:54.036625 3074 sys_catalog.cc:455] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } } }
I20260806 21:06:54.036800 3074 sys_catalog.cc:458] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:06:54.037441 3068 sys_catalog.cc:564] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 21:06:54.042701 3085 catalog_manager.cc:1558] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:06:54.042807 3085 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:06:54.046782 2913 master_runner.cc:428] Detected that this master db766e7b4c4b4ae1ba460ea4cf5a6131 is joining an existing cluster
I20260806 21:06:54.046850 2913 master_runner.cc:432] Initiating AddMaster RPC to add 127.27.225.252:34577
I20260806 21:06:54.053375 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 2403
I20260806 21:06:54.065217 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:43373
--local_ip_for_outbound_sockets=127.27.225.193
--tserver_master_addrs=127.27.225.254:45179
--webserver_port=42619
--webserver_interface=127.27.225.193
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.193.keytab
--principal=oryx/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:46765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.27.225.254:45179,127.27.225.253:34129
--tserver_master_addrs=127.27.225.254:45179,127.27.225.253:34129,127.27.225.252:34577 with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:06:54.092039 3094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.049545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:34129 (local address 127.0.0.1:49304)
0806 21:06:54.050053 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:54.050069 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:54.050210 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:54.050559 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:54.050561 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:54.050957 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:54.051191 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.051196 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.052272 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.052275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.053315 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.053325 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.054687 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.054690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.054820 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.055440 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:54.055452 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:54.087166 (+ 31714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:54.089764 (+ 2598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.089771 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.089773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.090107 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.090460 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.090463 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.090465 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.090564 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.091281 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:54.091286 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:54.091401 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 21:06:54.091599 (+ 198us) client_negotiation.cc:240] Negotiation successful
0806 21:06:54.091795 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"mutex_wait_us":92,"spinlock_wait_cycles":6528,"thread_start_us":146,"threads_started":1}
I20260806 21:06:54.092670 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.049290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49304 (local address 127.27.225.253:34129)
0806 21:06:54.049434 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.049439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.050147 (+ 708us) server_negotiation.cc:407] Connection header received
0806 21:06:54.050315 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.050318 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.050371 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.050425 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:54.051324 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.052139 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.053456 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.053853 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.053931 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.087404 (+ 33473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:54.087409 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:54.087412 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:54.087450 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:54.089587 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.090248 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.090252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.090253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.090349 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.090709 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.090714 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.090718 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.091047 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:54.091155 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.092299 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.092468 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 21:06:54.093483 3070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.049707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55050 (local address 127.27.225.252:34577)
0806 21:06:54.049841 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.049845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.050138 (+ 293us) server_negotiation.cc:407] Connection header received
0806 21:06:54.050305 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.050308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.050346 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.050396 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:54.051309 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.052139 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.053582 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.053930 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.053990 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.087561 (+ 33571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:54.087564 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:54.087566 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:54.087592 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:54.090306 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.092213 (+ 1907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.092220 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.092225 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.092303 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.092593 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.092596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.092598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.092800 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:54.092878 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.093235 (+ 357us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.093351 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 21:06:54.093991 3092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.049684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:34577 (local address 127.0.0.1:55050)
0806 21:06:54.050057 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:54.050070 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:54.050207 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:54.050533 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:54.050537 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:54.050964 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:54.051187 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.051196 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.052272 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.052275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.053484 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.053491 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.054165 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.054171 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.054301 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.054949 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:54.054968 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:54.087433 (+ 32465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:54.091866 (+ 4433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.091870 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.091872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.092095 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.092404 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.092417 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.092421 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.092502 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.092941 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:54.092944 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:54.093041 (+ 97us) client_negotiation.cc:769] Sending connection context
0806 21:06:54.093674 (+ 633us) client_negotiation.cc:240] Negotiation successful
0806 21:06:54.093811 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":21,"spinlock_wait_cycles":1280,"thread_start_us":85,"threads_started":1}
I20260806 21:06:54.096033 3093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.049147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.0.0.1:52576)
0806 21:06:54.049986 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:54.050001 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:54.050179 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:54.050571 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:54.050574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:54.050944 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:54.051182 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.051191 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.064863 (+ 13672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.064867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.065849 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.065867 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.066800 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.066804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.066918 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.067641 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:54.067663 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:54.087263 (+ 19600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:54.094232 (+ 6969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.094238 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.094243 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.094481 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.094919 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.094925 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.094928 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.095015 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.095460 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:54.095463 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:54.095596 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 21:06:54.095718 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 21:06:54.095863 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"mutex_wait_us":55,"thread_start_us":150,"threads_started":1}
I20260806 21:06:54.096066 2380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.049231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52576 (local address 127.27.225.254:45179)
0806 21:06:54.049371 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.049376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.050078 (+ 702us) server_negotiation.cc:407] Connection header received
0806 21:06:54.050306 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.050309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.050364 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.050420 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:54.051336 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.052180 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.066046 (+ 13866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.066439 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.066555 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.088026 (+ 21471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:54.088032 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:54.088039 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:54.088082 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:54.091734 (+ 3652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.094673 (+ 2939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.094677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.094679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.094794 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.095139 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.095142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.095143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.095345 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:54.095519 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.095797 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.095921 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 21:06:54 dist-test-slave-69rg krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050412, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.254@KRBTEST.COM for oryx/127.27.225.252@KRBTEST.COM
I20260806 21:06:54.108986 3070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.099844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55060 (local address 127.27.225.252:34577)
0806 21:06:54.099952 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.099956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.100037 (+ 81us) server_negotiation.cc:407] Connection header received
0806 21:06:54.100220 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.100223 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.100282 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.100371 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:54.100832 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.101504 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.102404 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.102712 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.102830 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.105386 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:54.105390 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:54.105392 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:54.105418 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:54.107102 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.107617 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.107620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.107622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.107714 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.108014 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.108018 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.108020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.108279 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:54.108393 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.108681 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.108811 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 21:06:54.108986 2903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.099800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:34577 (local address 127.0.0.1:55060)
0806 21:06:54.099980 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:54.099993 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:54.100084 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:54.100391 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:54.100394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:54.100611 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:54.100711 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.100718 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.101615 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.101619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.102279 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.102289 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.102999 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.103004 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.103138 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.103789 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:54.103805 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:54.105207 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:54.107251 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.107255 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.107258 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.107490 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.107815 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.107818 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.107820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.107915 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.108385 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:54.108389 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:54.108502 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 21:06:54.108624 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 21:06:54.108812 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
I20260806 21:06:54.110148 2338 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.27.225.252:34577: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131"
member_type: NON_VOTER
last_known_addr {
host: "127.27.225.252"
port: 34577
}
attrs {
promote: true
}
}
dest_uuid: "00de4fec8340498da672d706bcc37acf"
cas_config_opid_index: 5
I20260806 21:06:54.110829 2338 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: NON_VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: true } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: NON_VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: true } }
I20260806 21:06:54.112485 2907 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER db766e7b4c4b4ae1ba460ea4cf5a6131 (127.27.225.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: NON_VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: true } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: NON_VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: true } } }
I20260806 21:06:54.114118 2907 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.27.225.252:34577
I20260806 21:06:54.114429 2907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "00de4fec8340498da672d706bcc37acf" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: NON_VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: true } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: NON_VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: true } } }
I20260806 21:06:54.114548 2907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:54.115783 2913 master.cc:537] Master@127.27.225.252:34577 shutting down...
I20260806 21:06:54.116257 3042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:54.117738 3042 raft_consensus.cc:1261] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [term 1 FOLLOWER]: Refusing update from remote peer 00de4fec8340498da672d706bcc37acf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 21:06:54.118619 2906 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [LEADER]: Connected to new peer: Peer: permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: NON_VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 21:06:54.120661 2325 consensus_peers.cc:487] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf -> Peer db766e7b4c4b4ae1ba460ea4cf5a6131 (127.27.225.252:34577): Couldn't send request to peer db766e7b4c4b4ae1ba460ea4cf5a6131. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 21:06:54.123595 2913 raft_consensus.cc:2229] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:06:54.123889 2913 raft_consensus.cc:2258] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:54.123996 2913 tablet_replica.cc:330] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131: stopping tablet replica
I20260806 21:06:54.126710 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.115101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49314 (local address 127.27.225.253:34129)
0806 21:06:54.115235 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.115241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.121466 (+ 6225us) server_negotiation.cc:407] Connection header received
0806 21:06:54.121538 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.121541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.121606 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.121714 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:54.122531 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.123451 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.125225 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.125936 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.126071 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.126193 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.126496 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.126575 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 21:06:54.126725 2903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.115026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:34129 (local address 127.0.0.1:49314)
0806 21:06:54.119783 (+ 4757us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:54.119801 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:54.119897 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:54.121885 (+ 1988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:54.121890 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:54.121915 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:54.122008 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.122018 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.123600 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.123605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.124870 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.124883 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.126255 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.126257 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.126330 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.126369 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 21:06:54.126477 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 21:06:54.126569 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4684,"thread_start_us":102,"threads_started":1}
I20260806 21:06:54.127650 2957 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 1 LEARNER]: Refusing update from remote peer 00de4fec8340498da672d706bcc37acf: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 21:06:54.128290 3096 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: NON_VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 21:06:54.130646 2957 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER db766e7b4c4b4ae1ba460ea4cf5a6131 (127.27.225.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: NON_VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: true } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: NON_VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: true } } }
I20260806 21:06:54.131441 2913 master.cc:552] Master@127.27.225.252:34577 shutdown complete.
I20260806 21:06:54.131485 2913 master_runner.cc:305] Clearing existing system tablet
I20260806 21:06:54.131736 2913 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:54.131783 2913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:54.133044 3099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "00de4fec8340498da672d706bcc37acf" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: NON_VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: true } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: NON_VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: true } } }
I20260806 21:06:54.133203 3099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [sys.catalog]: This master's current role is: LEARNER
I20260806 21:06:54.133279 2986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00de4fec8340498da672d706bcc37acf. Latest consensus state: current_term: 1 leader_uuid: "00de4fec8340498da672d706bcc37acf" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: NON_VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: true } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: NON_VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: true } } }
I20260806 21:06:54.133376 2986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [sys.catalog]: This master's current role is: LEARNER
I20260806 21:06:54.134006 2913 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:06:54.134868 2670 heartbeater.cc:503] Master 127.27.225.254:45179 was elected leader, sending a full tablet report...
I20260806 21:06:54.135802 3101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:54.135983 2913 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:54.136114 2913 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 21:06:54.136152 2913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/wal
uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131"
format_stamp: "Formatted at 2026-08-06 21:06:53 on dist-test-slave-69rg"
server_key: "e8622e5bc5010d59d184df2aeaa258f9"
server_key_iv: "7263ef1667ad1aff07e1c1dda03b1340"
server_key_version: "encryptionkey@0"
I20260806 21:06:54.138252 2913 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:54.140798 2913 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 21:06:54.140921 2913 log.cc:1196] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 21:06:54.141180 2913 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131: Deleting consensus metadata
I20260806 21:06:54.141426 2913 master_runner.cc:315] Copying system tablet from 127.27.225.254:45179
I20260806 21:06:54.142689 2913 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131: tablet copy: Beginning tablet copy session from remote peer at address 127.27.225.254:45179
I20260806 21:06:54.143450 3096 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf: attempting to promote NON_VOTER 4859fba6ee99437aa8a0285c7270fe12 to VOTER
I20260806 21:06:54.144038 3096 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: NON_VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: true } }
I20260806 21:06:54.145499 2903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.145167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:34577 (local address 127.0.0.1:55074)
0806 21:06:54.145293 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:54.145384 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:34577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 21:06:54.145782 2957 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 1 LEARNER]: Refusing update from remote peer 00de4fec8340498da672d706bcc37acf: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 21:06:54.146320 2907 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 21:06:54.146746 2903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.146361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:34577 (local address 127.0.0.1:55080)
0806 21:06:54.146569 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:54.146631 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:34577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
W20260806 21:06:54.146945 2325 consensus_peers.cc:487] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf -> Peer db766e7b4c4b4ae1ba460ea4cf5a6131 (127.27.225.252:34577): Couldn't send request to peer db766e7b4c4b4ae1ba460ea4cf5a6131. Status: Network error: Client connection negotiation failed: client connection to 127.27.225.252:34577: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 21:06:54.148435 2906 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 4859fba6ee99437aa8a0285c7270fe12 (127.27.225.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: NON_VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: true } } }
I20260806 21:06:54.148967 2957 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 4859fba6ee99437aa8a0285c7270fe12 (127.27.225.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: NON_VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: true } } }
I20260806 21:06:54.150328 2906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "00de4fec8340498da672d706bcc37acf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: NON_VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: true } } }
I20260806 21:06:54.150451 2906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:54.150609 2906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "00de4fec8340498da672d706bcc37acf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: NON_VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: true } } }
I20260806 21:06:54.150642 2986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "00de4fec8340498da672d706bcc37acf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: NON_VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: true } } }
I20260806 21:06:54.150712 2906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:54.150749 2986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:06:54.158609 2380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.143787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52578 (local address 127.27.225.254:45179)
0806 21:06:54.143914 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.143919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.147756 (+ 3837us) server_negotiation.cc:407] Connection header received
0806 21:06:54.148061 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.148065 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.148117 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.148172 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:54.151811 (+ 3639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.152353 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.153235 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.153469 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.153584 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.155063 (+ 1479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:54.155066 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:54.155069 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:54.155097 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:54.156722 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.157175 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.157178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.157180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.157336 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.157638 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.157641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.157643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.157908 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:54.158026 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.158326 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.158441 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 21:06:54.158627 3107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.143339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.0.0.1:52578)
0806 21:06:54.147653 (+ 4314us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:54.147672 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:54.147893 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:54.151367 (+ 3474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:54.151369 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:54.151555 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:54.151679 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.151684 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.152482 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.152486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.153086 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.153095 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.153593 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.153597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.153695 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.154457 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:54.154490 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:54.154922 (+ 432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:54.156843 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.156848 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.156850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.157066 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.157439 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.157442 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.157444 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.157538 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.158015 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:54.158018 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:54.158136 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 21:06:54.158255 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 21:06:54.158428 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4230,"thread_start_us":90,"threads_started":1}
I20260806 21:06:54.159456 2361 tablet_copy_service.cc:140] P 00de4fec8340498da672d706bcc37acf: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer db766e7b4c4b4ae1ba460ea4cf5a6131 ({username='oryx', principal='oryx/127.27.225.252@KRBTEST.COM'} at 127.0.0.1:52578)
I20260806 21:06:54.159579 2361 tablet_copy_service.cc:161] P 00de4fec8340498da672d706bcc37acf: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer db766e7b4c4b4ae1ba460ea4cf5a6131 at {username='oryx', principal='oryx/127.27.225.252@KRBTEST.COM'} at 127.0.0.1:52578: session id = db766e7b4c4b4ae1ba460ea4cf5a6131-00000000000000000000000000000000
I20260806 21:06:54.160427 2361 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 21:06:54.161655 2913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 21:06:54.163664 2913 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131: tablet copy: Starting download of 0 data blocks...
I20260806 21:06:54.163884 2913 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131: tablet copy: Starting download of 1 WAL segments...
I20260806 21:06:54.165409 2913 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 21:06:54.167166 2361 tablet_copy_service.cc:342] P 00de4fec8340498da672d706bcc37acf: Request end of tablet copy session db766e7b4c4b4ae1ba460ea4cf5a6131-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.27.225.252@KRBTEST.COM'} at 127.0.0.1:52578
I20260806 21:06:54.167284 2361 tablet_copy_service.cc:433] P 00de4fec8340498da672d706bcc37acf: ending tablet copy session db766e7b4c4b4ae1ba460ea4cf5a6131-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer db766e7b4c4b4ae1ba460ea4cf5a6131
I20260806 21:06:54.174263 2913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:54.176002 3108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:54.176011 3111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:54.176625 3109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:54.178006 2913 server_base.cc:1034] running on GCE node
I20260806 21:06:54.178192 2913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:54.178537 2913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:54.179701 2913 hybrid_clock.cc:648] HybridClock initialized: now 1786050414179684 us; error 52 us; skew 500 ppm
Aug 06 21:06:54 dist-test-slave-69rg krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050414, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:54.181385 2913 init.cc:377] Logged in from keytab as oryx/127.27.225.252@KRBTEST.COM (short username oryx)
I20260806 21:06:54.181962 2913 webserver.cc:460] Webserver started at http://127.27.225.252:44319/ using document root <none> and password file <none>
I20260806 21:06:54.182181 2913 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:54.182245 2913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:54.183413 2913 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 21:06:54.184387 3118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:54.184623 2913 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 21:06:54.184721 2913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/wal
uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131"
format_stamp: "Formatted at 2026-08-06 21:06:53 on dist-test-slave-69rg"
server_key: "e8622e5bc5010d59d184df2aeaa258f9"
server_key_iv: "7263ef1667ad1aff07e1c1dda03b1340"
server_key_version: "encryptionkey@0"
I20260806 21:06:54.184815 2913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:54.189636 2913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:54.191707 2913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:54.196712 2913 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.252:34577
I20260806 21:06:54.196735 3171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.252:34577 every 8 connection(s)
I20260806 21:06:54.197315 2913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/master-2/data/info.pb
I20260806 21:06:54.199003 3172 sys_catalog.cc:263] Verifying existing consensus state
I20260806 21:06:54.199903 3172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131: Bootstrap starting.
I20260806 21:06:54.209769 3172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 21:06:54.210158 3172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131: Bootstrap complete.
I20260806 21:06:54.210832 3172 raft_consensus.cc:348] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: NON_VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: true } }
I20260806 21:06:54.211006 3172 raft_consensus.cc:729] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: db766e7b4c4b4ae1ba460ea4cf5a6131, State: Initialized, Role: LEARNER
I20260806 21:06:54.211127 3172 consensus_queue.cc:260] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: NON_VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: true } }
I20260806 21:06:54.211519 3175 sys_catalog.cc:455] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: NON_VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: true } } }
I20260806 21:06:54.211632 3172 sys_catalog.cc:564] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:06:54.211792 3175 sys_catalog.cc:458] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [sys.catalog]: This master's current role is: LEARNER
I20260806 21:06:54.215214 3186 catalog_manager.cc:1259] Loaded cluster ID: 42ffcff42f5a4ac2a89ff0d847fddd0b
I20260806 21:06:54.215343 3186 catalog_manager.cc:1552] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131: loading cluster ID for follower catalog manager: success
I20260806 21:06:54.217276 3186 catalog_manager.cc:1574] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131: acquiring CA information for follower catalog manager: success
I20260806 21:06:54.218704 3186 catalog_manager.cc:1602] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 21:06:54.236989 3095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:54.237378 3095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:54.237442 3095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:54.243690 3095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:54.243793 3095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:54.243884 3095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:06:54.252182 3095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:46765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.193.keytab
--principal=oryx/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:43373
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=42619
--tserver_master_addrs=127.27.225.254:45179,127.27.225.253:34129,127.27.225.252:34577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:54.253800 3095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:54.255128 3095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:54.265692 3192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:54.265792 3191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:54.266134 3194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:54.266183 3095 server_base.cc:1034] running on GCE node
I20260806 21:06:54.266628 3095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:54.267335 3095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:54.268529 3095 hybrid_clock.cc:648] HybridClock initialized: now 1786050414268499 us; error 50 us; skew 500 ppm
Aug 06 21:06:54 dist-test-slave-69rg krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050414, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:54.271265 3095 init.cc:377] Logged in from keytab as oryx/127.27.225.193@KRBTEST.COM (short username oryx)
I20260806 21:06:54.272239 3095 webserver.cc:460] Webserver started at http://127.27.225.193:42619/ using document root <none> and password file <none>
I20260806 21:06:54.272799 3095 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:54.272907 3095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:54.276870 3095 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:06:54.279095 3201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:54.280411 3095 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 21:06:54.280548 3095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "513daa018ebf4514894d5ef4d7cb795e"
format_stamp: "Formatted at 2026-08-06 21:06:52 on dist-test-slave-69rg"
server_key: "bfd194a904feb9857712eb2fcf30e764"
server_key_iv: "b85d8eac4c13ab6dcd7863ca394397cd"
server_key_version: "encryptionkey@0"
I20260806 21:06:54.281124 3095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:54.293809 3095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:54.296979 3095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:54.297196 3095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:54.297876 3095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:54.298801 3095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:54.298873 3095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:54.298944 3095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:54.298988 3095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:54.309549 3095 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:43373
I20260806 21:06:54.309571 3317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:43373 every 8 connection(s)
I20260806 21:06:54.310882 3095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:06:54.318801 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 3095
I20260806 21:06:54.319025 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 2539
Aug 06 21:06:54 dist-test-slave-69rg krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050414, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.253@KRBTEST.COM
Aug 06 21:06:54 dist-test-slave-69rg krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050414, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM
Aug 06 21:06:54 dist-test-slave-69rg krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050414, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.252@KRBTEST.COM
I20260806 21:06:54.329756 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.314638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:51799 (local address 127.27.225.253:34129)
0806 21:06:54.314793 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.314799 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.315073 (+ 274us) server_negotiation.cc:407] Connection header received
0806 21:06:54.315475 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.315480 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.315548 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.315621 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:54.317314 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.318181 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.320387 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.320720 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.320799 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.324935 (+ 4136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:54.324939 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:54.324942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:54.324975 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:54.327014 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.327751 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.327755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.327757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.327852 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.328346 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.328349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.328352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.328617 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:54.328692 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.329439 (+ 747us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.329579 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 21:06:54.329756 2380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.314099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:38979 (local address 127.27.225.254:45179)
0806 21:06:54.314249 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.314256 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.314312 (+ 56us) server_negotiation.cc:407] Connection header received
0806 21:06:54.315747 (+ 1435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.315750 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.315823 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.315928 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:54.317318 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.318076 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.320012 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.320375 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.320456 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.325317 (+ 4861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:54.325321 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:54.325323 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:54.325352 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:54.327738 (+ 2386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.328340 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.328344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.328345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.328437 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.328745 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.328750 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.328752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.328974 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:54.329043 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.329440 (+ 397us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.329578 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 21:06:54.331162 3324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.313371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:34129 (local address 127.27.225.193:51799)
0806 21:06:54.314959 (+ 1588us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:54.314974 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:54.315241 (+ 267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:54.315893 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:54.315905 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:54.316433 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:54.317142 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.317153 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.318299 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.318304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.320232 (+ 1928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.320241 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.320958 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.320961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.321045 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.321889 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:54.321913 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:54.324712 (+ 2799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:54.327174 (+ 2462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.327183 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.327204 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.327618 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.327967 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.327971 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.327973 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.328223 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.328775 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:54.328782 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:54.329134 (+ 352us) client_negotiation.cc:769] Sending connection context
0806 21:06:54.330037 (+ 903us) client_negotiation.cc:240] Negotiation successful
0806 21:06:54.330182 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1475,"mutex_wait_us":189,"spinlock_wait_cycles":23936,"thread_start_us":198,"threads_started":1}
I20260806 21:06:54.331605 3325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.314688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:47083 (local address 127.27.225.252:34577)
0806 21:06:54.316052 (+ 1364us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.316059 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.316075 (+ 16us) server_negotiation.cc:407] Connection header received
0806 21:06:54.316128 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.316133 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.316186 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.316294 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:54.317314 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.318536 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.320665 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.321181 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.321981 (+ 800us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.325625 (+ 3644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:54.325632 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:54.325639 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:54.325680 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:54.327804 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.328420 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.328426 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.328431 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.328529 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.328854 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.328860 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.328865 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.329121 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:54.329217 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.331289 (+ 2072us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.331443 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1282,"thread_start_us":80,"threads_started":1}
I20260806 21:06:54.331173 3323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.314587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:34577 (local address 127.27.225.193:47083)
0806 21:06:54.315663 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:54.315676 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:54.315757 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:54.316438 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:54.316441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:54.316611 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:54.317142 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.317153 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.318753 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.318757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.320488 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.320504 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.322141 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.322149 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.322308 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.322893 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:54.322915 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:54.325136 (+ 2221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:54.327953 (+ 2817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.327960 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.327965 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.328291 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.328640 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.328646 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.328651 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.328750 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.329274 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:54.329280 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:54.329390 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 21:06:54.329516 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 21:06:54.329955 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1014,"mutex_wait_us":87,"thread_start_us":100,"threads_started":1}
I20260806 21:06:54.332234 3322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.313371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.193:38979)
0806 21:06:54.314010 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:54.314067 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:54.315241 (+ 1174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:54.315945 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:54.315948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:54.316417 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:54.317131 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.317148 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.318233 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.318236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.319860 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.319872 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.320681 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.320685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.320931 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.321920 (+ 989us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:54.321931 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:54.324715 (+ 2784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:54.327877 (+ 3162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.327883 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.327886 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.328205 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.328553 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.328555 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.328557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.328637 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.329118 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:54.329121 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:54.329190 (+ 69us) client_negotiation.cc:769] Sending connection context
0806 21:06:54.330437 (+ 1247us) client_negotiation.cc:240] Negotiation successful
0806 21:06:54.330578 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":95,"spinlock_wait_cycles":9984,"thread_start_us":762,"threads_started":1}
I20260806 21:06:54.332844 3320 heartbeater.cc:347] Connected to a master server at 127.27.225.253:34129
I20260806 21:06:54.333086 3320 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:54.333421 3319 heartbeater.cc:347] Connected to a master server at 127.27.225.254:45179
I20260806 21:06:54.333503 3319 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:54.333751 3320 heartbeater.cc:511] Master 127.27.225.253:34129 requested a full tablet report, sending...
I20260806 21:06:54.333752 3319 heartbeater.cc:511] Master 127.27.225.254:45179 requested a full tablet report, sending...
I20260806 21:06:54.334118 3318 heartbeater.cc:347] Connected to a master server at 127.27.225.252:34577
I20260806 21:06:54.334208 3318 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:54.334378 3318 heartbeater.cc:511] Master 127.27.225.252:34577 requested a full tablet report, sending...
I20260806 21:06:54.335412 2947 ts_manager.cc:194] Registered new tserver with Master: 513daa018ebf4514894d5ef4d7cb795e (127.27.225.193:43373)
I20260806 21:06:54.335583 3135 ts_manager.cc:194] Registered new tserver with Master: 513daa018ebf4514894d5ef4d7cb795e (127.27.225.193:43373)
I20260806 21:06:54.336367 2338 ts_manager.cc:194] Re-registered known tserver with Master: 513daa018ebf4514894d5ef4d7cb795e (127.27.225.193:43373)
I20260806 21:06:54.337100 2338 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.225.193@KRBTEST.COM'} at 127.27.225.193:38979
I20260806 21:06:54.337204 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:42173
--local_ip_for_outbound_sockets=127.27.225.194
--tserver_master_addrs=127.27.225.254:45179
--webserver_port=44841
--webserver_interface=127.27.225.194
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.194.keytab
--principal=oryx/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:46765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.27.225.254:45179,127.27.225.253:34129
--tserver_master_addrs=127.27.225.254:45179,127.27.225.253:34129,127.27.225.252:34577 with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:06:54.368060 2805 heartbeater.cc:503] Master 127.27.225.254:45179 was elected leader, sending a full tablet report...
W20260806 21:06:54.451362 3326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:54.451676 3326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:54.451756 3326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:54.455895 3326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:54.455981 3326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:54.456070 3326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:06:54.461386 3326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:46765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.194.keytab
--principal=oryx/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:42173
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=44841
--tserver_master_addrs=127.27.225.254:45179,127.27.225.253:34129,127.27.225.252:34577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:54.462597 3326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:54.463614 3326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:54.473289 3331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:54.473272 3332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:54.473272 3334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:54.473614 3326 server_base.cc:1034] running on GCE node
I20260806 21:06:54.474143 3326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:54.474839 3326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:54.476029 3326 hybrid_clock.cc:648] HybridClock initialized: now 1786050414476015 us; error 25 us; skew 500 ppm
Aug 06 21:06:54 dist-test-slave-69rg krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050414, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:54.478857 3326 init.cc:377] Logged in from keytab as oryx/127.27.225.194@KRBTEST.COM (short username oryx)
I20260806 21:06:54.479756 3326 webserver.cc:460] Webserver started at http://127.27.225.194:44841/ using document root <none> and password file <none>
I20260806 21:06:54.480315 3326 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:54.480423 3326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:54.484259 3326 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 21:06:54.486404 3341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:54.487641 3326 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:06:54.487741 3326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "130105c8e4b94acea85e45c8cbbfd8ad"
format_stamp: "Formatted at 2026-08-06 21:06:53 on dist-test-slave-69rg"
server_key: "1a8f859815d4822c99f003b5e054adb2"
server_key_iv: "b3f41df1f43f852bd75266ca3eeecf51"
server_key_version: "encryptionkey@0"
I20260806 21:06:54.488238 3326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:54.506747 3326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:54.510281 3326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:54.510448 3326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:54.511060 3326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:54.512130 3326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:54.512214 3326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:54.512270 3326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:54.512298 3326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:54.522068 3326 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:42173
I20260806 21:06:54.522110 3455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:42173 every 8 connection(s)
I20260806 21:06:54.523126 3326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:06:54.525152 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 3326
I20260806 21:06:54.525326 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 2675
I20260806 21:06:54.533280 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:43009
--local_ip_for_outbound_sockets=127.27.225.195
--tserver_master_addrs=127.27.225.254:45179
--webserver_port=36289
--webserver_interface=127.27.225.195
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.195.keytab
--principal=oryx/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:46765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.27.225.254:45179,127.27.225.253:34129
--tserver_master_addrs=127.27.225.254:45179,127.27.225.253:34129,127.27.225.252:34577 with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 21:06:54 dist-test-slave-69rg krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050414, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM
Aug 06 21:06:54 dist-test-slave-69rg krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050414, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.252@KRBTEST.COM
Aug 06 21:06:54 dist-test-slave-69rg krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050414, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.253@KRBTEST.COM
I20260806 21:06:54.542701 2380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.525191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:54833 (local address 127.27.225.254:45179)
0806 21:06:54.525457 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.525462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.526845 (+ 1383us) server_negotiation.cc:407] Connection header received
0806 21:06:54.528325 (+ 1480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.528329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.528388 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.528470 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:54.530533 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.531490 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.533298 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.533664 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.533805 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.538495 (+ 4690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:54.538502 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:54.538508 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:54.538551 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:54.540551 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.541010 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.541013 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.541015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.541106 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.541399 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.541403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.541405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.541610 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:54.541727 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.542378 (+ 651us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.542520 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194}
I20260806 21:06:54.543124 3325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.527205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:57313 (local address 127.27.225.252:34577)
0806 21:06:54.529062 (+ 1857us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.529067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.529634 (+ 567us) server_negotiation.cc:407] Connection header received
0806 21:06:54.529804 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.529810 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.529901 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.529994 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:54.530532 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.531371 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.533902 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.534245 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.534383 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.538505 (+ 4122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:54.538508 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:54.538510 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:54.538537 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:54.540155 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.540788 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.540795 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.540800 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.540912 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.541414 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.541421 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.541424 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.541696 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:54.541843 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.542708 (+ 865us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.542890 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1739}
I20260806 21:06:54.544090 3463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.526848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:34577 (local address 127.27.225.194:57313)
0806 21:06:54.529543 (+ 2695us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:54.529559 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:54.529703 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:54.530022 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:54.530025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:54.530217 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:54.530395 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.530403 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.531569 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.531572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.533144 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.533157 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.534614 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.534621 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.534752 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.535460 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:54.535475 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:54.538266 (+ 2791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:54.540310 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.540317 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.540337 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.540670 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.541046 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.541052 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.541056 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.541282 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.541898 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:54.541903 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:54.542106 (+ 203us) client_negotiation.cc:769] Sending connection context
0806 21:06:54.542294 (+ 188us) client_negotiation.cc:240] Negotiation successful
0806 21:06:54.542761 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2633,"mutex_wait_us":260,"spinlock_wait_cycles":8192,"thread_start_us":82,"threads_started":1}
I20260806 21:06:54.544677 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.525569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.194:54833)
0806 21:06:54.526671 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:54.526730 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:54.528074 (+ 1344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:54.528664 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:54.528670 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:54.529403 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:54.530338 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.530362 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.531656 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.531662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.533137 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.533147 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.533877 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.533883 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.534088 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.534824 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:54.534848 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:54.538266 (+ 3418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:54.540678 (+ 2412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.540681 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.540682 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.540894 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.541208 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.541211 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.541213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.541297 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.541725 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:54.541733 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:54.542092 (+ 359us) client_negotiation.cc:769] Sending connection context
0806 21:06:54.542294 (+ 202us) client_negotiation.cc:240] Negotiation successful
0806 21:06:54.544469 (+ 2175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":854,"mutex_wait_us":530,"spinlock_wait_cycles":34432,"thread_start_us":320,"threads_started":1}
I20260806 21:06:54.545454 3457 heartbeater.cc:347] Connected to a master server at 127.27.225.252:34577
I20260806 21:06:54.545547 3458 heartbeater.cc:347] Connected to a master server at 127.27.225.254:45179
I20260806 21:06:54.545709 3458 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:54.545710 3457 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:54.546435 3458 heartbeater.cc:511] Master 127.27.225.254:45179 requested a full tablet report, sending...
I20260806 21:06:54.546435 3457 heartbeater.cc:511] Master 127.27.225.252:34577 requested a full tablet report, sending...
I20260806 21:06:54.548099 3135 ts_manager.cc:194] Registered new tserver with Master: 130105c8e4b94acea85e45c8cbbfd8ad (127.27.225.194:42173)
I20260806 21:06:54.548116 2338 ts_manager.cc:194] Re-registered known tserver with Master: 130105c8e4b94acea85e45c8cbbfd8ad (127.27.225.194:42173)
I20260806 21:06:54.548888 2338 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.225.194@KRBTEST.COM'} at 127.27.225.194:54833
I20260806 21:06:54.550344 3462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.526416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:34129 (local address 127.27.225.194:43979)
0806 21:06:54.528065 (+ 1649us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:54.528079 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:54.528224 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:54.528661 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:54.528668 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:54.529403 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:54.530340 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.530363 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.531569 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.531572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.533899 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.533913 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.534774 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.534779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.534912 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.535468 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:54.535482 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:54.546260 (+ 10778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:54.548575 (+ 2315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.548582 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.548585 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.548822 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.549154 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.549160 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.549164 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.549274 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.549740 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:54.549743 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:54.549849 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 21:06:54.549953 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 21:06:54.550163 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1587,"mutex_wait_us":1119,"thread_start_us":91,"threads_started":1}
I20260806 21:06:54.550371 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.525160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:43979 (local address 127.27.225.253:34129)
0806 21:06:54.525361 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.525366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.528190 (+ 2824us) server_negotiation.cc:407] Connection header received
0806 21:06:54.528353 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.528356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.528407 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.528465 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:54.530532 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.531371 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.534056 (+ 2685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.534471 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.534569 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.546457 (+ 11888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:54.546464 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:54.546470 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:54.546503 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:54.548468 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.548942 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.548945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.548947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.549039 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.549386 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.549391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.549394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.549629 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:54.549728 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.550049 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.550199 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 21:06:54.550985 3459 heartbeater.cc:347] Connected to a master server at 127.27.225.253:34129
I20260806 21:06:54.551074 3459 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:54.551249 3459 heartbeater.cc:511] Master 127.27.225.253:34129 requested a full tablet report, sending...
I20260806 21:06:54.551885 2947 ts_manager.cc:194] Registered new tserver with Master: 130105c8e4b94acea85e45c8cbbfd8ad (127.27.225.194:42173)
I20260806 21:06:54.606925 2903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.603496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:34577 (local address 127.0.0.1:55090)
0806 21:06:54.603650 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:54.603674 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:54.603806 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:54.604095 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:54.604098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:54.604118 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:54.604232 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.604238 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.604998 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.605003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.605757 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.605766 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.606510 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.606514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.606599 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.606623 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 21:06:54.606701 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 21:06:54.606782 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 21:06:54.606925 3325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.603671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55090 (local address 127.27.225.252:34577)
0806 21:06:54.603825 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.603830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.603864 (+ 34us) server_negotiation.cc:407] Connection header received
0806 21:06:54.603928 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.603930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.603980 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.604079 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:54.604354 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.604878 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.605936 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.606376 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.606513 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.606674 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.606714 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.606780 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 21:06:54.607635 3145 raft_consensus.cc:1203] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 21:06:54.608052 3175 sys_catalog.cc:455] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00de4fec8340498da672d706bcc37acf. Latest consensus state: current_term: 1 leader_uuid: "00de4fec8340498da672d706bcc37acf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: NON_VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: true } } }
I20260806 21:06:54.608232 3175 sys_catalog.cc:458] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [sys.catalog]: This master's current role is: LEARNER
W20260806 21:06:54.654110 3464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:06:54.654397 3464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:06:54.654443 3464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:06:54.658772 3464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:06:54.658849 3464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:06:54.658911 3464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:06:54.664501 3464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:46765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.195.keytab
--principal=oryx/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:43009
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=36289
--tserver_master_addrs=127.27.225.254:45179,127.27.225.253:34129,127.27.225.252:34577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:06:54.665550 3464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:06:54.666632 3464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:06:54.676783 3470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:54.676762 3469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:06:54.676762 3472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:06:54.677285 3464 server_base.cc:1034] running on GCE node
I20260806 21:06:54.677738 3464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:06:54.678467 3464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:06:54.679662 3464 hybrid_clock.cc:648] HybridClock initialized: now 1786050414679653 us; error 29 us; skew 500 ppm
Aug 06 21:06:54 dist-test-slave-69rg krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050414, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:06:54.682883 3464 init.cc:377] Logged in from keytab as oryx/127.27.225.195@KRBTEST.COM (short username oryx)
I20260806 21:06:54.683698 3464 webserver.cc:460] Webserver started at http://127.27.225.195:36289/ using document root <none> and password file <none>
I20260806 21:06:54.684260 3464 fs_manager.cc:359] Metadata directory not provided
I20260806 21:06:54.684330 3464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:06:54.688483 3464 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260806 21:06:54.690577 3479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:54.692106 3464 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:06:54.692217 3464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "3ae6436ae2ee446eacad8af409ce0daf"
format_stamp: "Formatted at 2026-08-06 21:06:53 on dist-test-slave-69rg"
server_key: "79083649b13da6e93800e1fb3b6c971a"
server_key_iv: "2c2a265030cd44c8ad601ef3cb4f096d"
server_key_version: "encryptionkey@0"
I20260806 21:06:54.692777 3464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:06:54.710461 3464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:06:54.714082 3464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:06:54.714304 3464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:06:54.714993 3464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:06:54.716029 3464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:06:54.716100 3464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:54.716171 3464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:06:54.716223 3464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:06:54.726337 3464 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:43009
I20260806 21:06:54.726356 3593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:43009 every 8 connection(s)
I20260806 21:06:54.727444 3464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:06:54.732802 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 3464
Aug 06 21:06:54 dist-test-slave-69rg krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050414, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM
Aug 06 21:06:54 dist-test-slave-69rg krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050414, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.252@KRBTEST.COM
Aug 06 21:06:54 dist-test-slave-69rg krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050414, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.253@KRBTEST.COM
I20260806 21:06:54.747745 3325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.731072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:48669 (local address 127.27.225.252:34577)
0806 21:06:54.731207 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.731210 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.731264 (+ 54us) server_negotiation.cc:407] Connection header received
0806 21:06:54.732503 (+ 1239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.732507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.732563 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.732642 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:54.733927 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.734759 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.736791 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.737147 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.737221 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.742506 (+ 5285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:54.742512 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:54.742516 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:54.742546 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:54.744497 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.745547 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.745552 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.745554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.745688 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.746195 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.746198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.746199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.746493 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:54.746607 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.747408 (+ 801us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.747548 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 21:06:54.747830 2380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.729376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:48467 (local address 127.27.225.254:45179)
0806 21:06:54.729516 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.729521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.730516 (+ 995us) server_negotiation.cc:407] Connection header received
0806 21:06:54.731844 (+ 1328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.731848 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.731909 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.732014 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:54.733924 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.734759 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.736664 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.736987 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.737078 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.740728 (+ 3650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:54.740731 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:54.740735 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:54.740764 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:54.742663 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.746249 (+ 3586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.746254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.746257 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.746364 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.746739 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.746745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.746748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.746990 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:54.747081 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.747479 (+ 398us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.747638 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 21:06:54.748853 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.729304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:34765 (local address 127.27.225.253:34129)
0806 21:06:54.729467 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.729472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.730706 (+ 1234us) server_negotiation.cc:407] Connection header received
0806 21:06:54.732212 (+ 1506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.732216 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.732270 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.732353 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:54.733930 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.734694 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.736599 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.736950 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.737037 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.745155 (+ 8118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:54.745161 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:54.745165 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:54.745194 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:54.746959 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.747465 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.747468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.747470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.747560 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.747883 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.747886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.747888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.748153 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:54.748230 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.748563 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.748684 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 21:06:54.750015 3599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.729737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:34129 (local address 127.27.225.195:34765)
0806 21:06:54.730600 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:54.730615 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:54.731585 (+ 970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:54.732407 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:54.732410 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:54.732931 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:54.733717 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.733739 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.734878 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.734881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.736458 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.736472 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.737257 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.737261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.737485 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.738632 (+ 1147us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:54.738654 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:54.744988 (+ 6334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:54.747082 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.747085 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.747087 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.747357 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.747671 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.747674 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.747676 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.747769 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.748288 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:54.748291 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:54.748377 (+ 86us) client_negotiation.cc:769] Sending connection context
0806 21:06:54.749063 (+ 686us) client_negotiation.cc:240] Negotiation successful
0806 21:06:54.749207 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":778,"mutex_wait_us":154,"spinlock_wait_cycles":17792,"thread_start_us":123,"threads_started":1}
I20260806 21:06:54.750072 3601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.730903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:34577 (local address 127.27.225.195:48669)
0806 21:06:54.731189 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:54.731205 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:54.731585 (+ 380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:54.732773 (+ 1188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:54.732776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:54.732991 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:54.733717 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.733738 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.734906 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.734910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.736676 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.736684 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.737374 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.737379 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.737540 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.738478 (+ 938us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:54.738502 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:54.742339 (+ 3837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:54.744645 (+ 2306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.744649 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.744651 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.745432 (+ 781us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.745845 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.745851 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.745854 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.746063 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.746671 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:54.746682 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:54.747067 (+ 385us) client_negotiation.cc:769] Sending connection context
0806 21:06:54.748379 (+ 1312us) client_negotiation.cc:240] Negotiation successful
0806 21:06:54.748544 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":660,"spinlock_wait_cycles":6144,"thread_start_us":106,"threads_started":1}
I20260806 21:06:54.751389 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.737237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49322 (local address 127.27.225.253:34129)
0806 21:06:54.738002 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.738009 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.738149 (+ 140us) server_negotiation.cc:407] Connection header received
0806 21:06:54.738245 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.738247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.738335 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.738428 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:54.739435 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.740253 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.741502 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.742137 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.742285 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.744847 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:54.744851 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:54.744854 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:54.744884 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:54.748024 (+ 3140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.749726 (+ 1702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.749729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.749731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.749836 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.750151 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.750155 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.750157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.750352 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:54.750453 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.751072 (+ 619us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.751222 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":78,"threads_started":1}
I20260806 21:06:54.751443 3596 heartbeater.cc:347] Connected to a master server at 127.27.225.253:34129
I20260806 21:06:54.751578 3608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.735947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55092 (local address 127.27.225.252:34577)
0806 21:06:54.736992 (+ 1045us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.736998 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.737634 (+ 636us) server_negotiation.cc:407] Connection header received
0806 21:06:54.737679 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.737682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.737741 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.738618 (+ 877us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:54.739666 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.740428 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.741721 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.742187 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.742293 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.744977 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:54.744982 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:54.744984 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:54.745010 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:54.747216 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.748547 (+ 1331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.748550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.748552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.748639 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.749277 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.749281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.749282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.749489 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:54.749611 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.751220 (+ 1609us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.751344 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":966,"thread_start_us":70,"threads_started":1}
I20260806 21:06:54.751721 3596 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:54.752362 3596 heartbeater.cc:511] Master 127.27.225.253:34129 requested a full tablet report, sending...
I20260806 21:06:54.751443 3594 heartbeater.cc:347] Connected to a master server at 127.27.225.252:34577
I20260806 21:06:54.753021 3594 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:54.753203 3594 heartbeater.cc:511] Master 127.27.225.252:34577 requested a full tablet report, sending...
I20260806 21:06:54.750015 3600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.729737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.195:48467)
0806 21:06:54.730344 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:54.730400 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:54.731617 (+ 1217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:54.732253 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:54.732262 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:54.732927 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:06:54.733717 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.733738 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.734906 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.734910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.736458 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.736467 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.737416 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.737422 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.737570 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.738516 (+ 946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:06:54.738531 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:06:54.740555 (+ 2024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:06:54.743442 (+ 2887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.743457 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.744891 (+ 1434us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.745215 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.746491 (+ 1276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:06:54.746496 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:06:54.746500 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:06:54.746595 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:06:54.747156 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:06:54.747159 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:06:54.747264 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 21:06:54.748059 (+ 795us) client_negotiation.cc:240] Negotiation successful
0806 21:06:54.748458 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":533,"spinlock_wait_cycles":28672,"thread_start_us":161,"threads_started":1}
I20260806 21:06:54.753567 2947 ts_manager.cc:194] Registered new tserver with Master: 3ae6436ae2ee446eacad8af409ce0daf (127.27.225.195:43009)
I20260806 21:06:54.754410 3135 ts_manager.cc:194] Registered new tserver with Master: 3ae6436ae2ee446eacad8af409ce0daf (127.27.225.195:43009)
I20260806 21:06:54.754623 3595 heartbeater.cc:347] Connected to a master server at 127.27.225.254:45179
I20260806 21:06:54.754735 3595 heartbeater.cc:464] Registering TS with master...
I20260806 21:06:54.754901 3595 heartbeater.cc:511] Master 127.27.225.254:45179 requested a full tablet report, sending...
I20260806 21:06:54.755631 2338 ts_manager.cc:194] Re-registered known tserver with Master: 3ae6436ae2ee446eacad8af409ce0daf (127.27.225.195:43009)
I20260806 21:06:54.756516 2338 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.225.195@KRBTEST.COM'} at 127.27.225.195:48467
I20260806 21:06:54.759320 3607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.735361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52584 (local address 127.27.225.254:45179)
0806 21:06:54.736427 (+ 1066us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.736432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.737840 (+ 1408us) server_negotiation.cc:407] Connection header received
0806 21:06:54.737892 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.737896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.737946 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.738034 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:54.740509 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.749504 (+ 8995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.752312 (+ 2808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.752658 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.752745 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.755289 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:54.755293 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:54.755299 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:54.755329 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:54.757319 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.757865 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.757869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.757871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.757967 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:54.758291 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:54.758294 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:54.758296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:54.758560 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:54.758715 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.759006 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.759138 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":986,"thread_start_us":79,"threads_started":1}
I20260806 21:06:54.762151 2338 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52584:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:06:54.765323 2338 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:06:54.768167 2907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "00de4fec8340498da672d706bcc37acf" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: NON_VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: true } } }
I20260806 21:06:54.768343 2907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:54.771957 2907 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf: attempting to promote NON_VOTER db766e7b4c4b4ae1ba460ea4cf5a6131 to VOTER
I20260806 21:06:54.772292 2907 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: false } }
I20260806 21:06:54.773425 2957 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 1 FOLLOWER]: Refusing update from remote peer 00de4fec8340498da672d706bcc37acf: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 21:06:54.774233 3145 raft_consensus.cc:1261] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [term 1 LEARNER]: Refusing update from remote peer 00de4fec8340498da672d706bcc37acf: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 21:06:54.774211 3615 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 21:06:54.774610 3615 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [LEADER]: Connected to new peer: Peer: permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 21:06:54.776120 3613 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, db766e7b4c4b4ae1ba460ea4cf5a6131 (127.27.225.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: false } } }
I20260806 21:06:54.776757 2957 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, db766e7b4c4b4ae1ba460ea4cf5a6131 (127.27.225.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: false } } }
I20260806 21:06:54.777034 3144 raft_consensus.cc:2932] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, db766e7b4c4b4ae1ba460ea4cf5a6131 (127.27.225.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: false } } }
I20260806 21:06:54.777877 3613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "00de4fec8340498da672d706bcc37acf" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: false } } }
I20260806 21:06:54.777976 3613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:54.778108 3613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "00de4fec8340498da672d706bcc37acf" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: false } } }
I20260806 21:06:54.778172 3613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:54.778258 3613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "00de4fec8340498da672d706bcc37acf" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: false } } }
I20260806 21:06:54.778326 3613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:54.779140 3623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "00de4fec8340498da672d706bcc37acf" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: false } } }
I20260806 21:06:54.779278 3623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:06:54.779913 3175 sys_catalog.cc:455] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "00de4fec8340498da672d706bcc37acf" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: false } } }
I20260806 21:06:54.780033 3175 sys_catalog.cc:458] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:06:54.787005 2903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.781592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:43373 (local address 127.0.0.1:54298)
0806 21:06:54.781971 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:54.781984 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:54.782053 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:54.782800 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:54.782803 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:54.782824 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:54.782941 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.782952 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.784328 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.784333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.785537 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.785570 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.786523 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.786527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.786656 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.786680 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 21:06:54.786763 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 21:06:54.786855 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":60}
I20260806 21:06:54.787031 3624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.781926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54298 (local address 127.27.225.193:43373)
0806 21:06:54.782227 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.782251 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.782279 (+ 28us) server_negotiation.cc:407] Connection header received
0806 21:06:54.782372 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.782378 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.782645 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.782853 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:54.783040 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.784185 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.785672 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.786377 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.786525 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.786684 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.786797 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.786895 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":125,"threads_started":1}
I20260806 21:06:54.789701 3627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.782424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:43009 (local address 127.0.0.1:51474)
0806 21:06:54.783249 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:54.783265 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:54.783364 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:54.784916 (+ 1552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:54.784919 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:54.784937 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:54.785021 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.785027 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.786543 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.786547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.787968 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.787978 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.789267 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.789270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.789372 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.789396 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 21:06:54.789461 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 21:06:54.789534 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"mutex_wait_us":3,"thread_start_us":107,"threads_started":1}
I20260806 21:06:54.789923 3625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.783009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:42173 (local address 127.0.0.1:34014)
0806 21:06:54.783851 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:54.783866 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:54.783967 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:54.784608 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:54.784611 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:54.784664 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:54.784838 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.784849 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.786602 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.786605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.788359 (+ 1754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.788370 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.789420 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.789424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.789516 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.789541 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 21:06:54.789622 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 21:06:54.789691 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":777,"mutex_wait_us":651,"thread_start_us":97,"threads_started":1}
I20260806 21:06:54.790186 3626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.782615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51474 (local address 127.27.225.195:43009)
0806 21:06:54.784290 (+ 1675us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.784320 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.784344 (+ 24us) server_negotiation.cc:407] Connection header received
0806 21:06:54.784417 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.784422 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.784601 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.784759 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:54.785156 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.786401 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.788106 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.788934 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.789071 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.789196 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.789925 (+ 729us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.790038 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1499,"thread_start_us":127,"threads_started":1}
I20260806 21:06:54.790232 3628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.783218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34014 (local address 127.27.225.194:42173)
0806 21:06:54.783692 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.783704 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.784028 (+ 324us) server_negotiation.cc:407] Connection header received
0806 21:06:54.784122 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.784128 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.784439 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.784992 (+ 553us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:54.785264 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.786485 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.788495 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.789287 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.789402 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.789527 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.789643 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.789738 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":121,"threads_started":1}
I20260806 21:06:54.790428 3250 tablet_service.cc:1467] Processing CreateTablet for tablet c020b47ed720452794e1e37ed8daf5db (DEFAULT_TABLE table=test-table [id=0ea1a35cd0df4103ad37b312efad49f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:54.791548 3250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c020b47ed720452794e1e37ed8daf5db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:54.792775 3528 tablet_service.cc:1467] Processing CreateTablet for tablet c020b47ed720452794e1e37ed8daf5db (DEFAULT_TABLE table=test-table [id=0ea1a35cd0df4103ad37b312efad49f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:54.793609 3528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c020b47ed720452794e1e37ed8daf5db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:54.796022 3390 tablet_service.cc:1467] Processing CreateTablet for tablet c020b47ed720452794e1e37ed8daf5db (DEFAULT_TABLE table=test-table [id=0ea1a35cd0df4103ad37b312efad49f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:54.797053 3390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c020b47ed720452794e1e37ed8daf5db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:54.799861 3629 tablet_bootstrap.cc:492] T c020b47ed720452794e1e37ed8daf5db P 513daa018ebf4514894d5ef4d7cb795e: Bootstrap starting.
I20260806 21:06:54.801011 3630 tablet_bootstrap.cc:492] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf: Bootstrap starting.
I20260806 21:06:54.802064 3629 tablet_bootstrap.cc:654] T c020b47ed720452794e1e37ed8daf5db P 513daa018ebf4514894d5ef4d7cb795e: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:54.802930 3629 log.cc:824] T c020b47ed720452794e1e37ed8daf5db P 513daa018ebf4514894d5ef4d7cb795e: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:54.803328 3630 tablet_bootstrap.cc:654] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:54.804066 3630 log.cc:824] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:54.806149 3630 tablet_bootstrap.cc:492] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf: No bootstrap required, opened a new log
I20260806 21:06:54.806360 3630 ts_tablet_manager.cc:1397] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260806 21:06:54.807480 3629 tablet_bootstrap.cc:492] T c020b47ed720452794e1e37ed8daf5db P 513daa018ebf4514894d5ef4d7cb795e: No bootstrap required, opened a new log
I20260806 21:06:54.807668 3629 ts_tablet_manager.cc:1397] T c020b47ed720452794e1e37ed8daf5db P 513daa018ebf4514894d5ef4d7cb795e: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260806 21:06:54.810882 3630 raft_consensus.cc:348] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513daa018ebf4514894d5ef4d7cb795e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43373 } } peers { permanent_uuid: "3ae6436ae2ee446eacad8af409ce0daf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43009 } } peers { permanent_uuid: "130105c8e4b94acea85e45c8cbbfd8ad" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42173 } }
I20260806 21:06:54.811154 3630 raft_consensus.cc:374] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:54.811204 3630 raft_consensus.cc:729] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae6436ae2ee446eacad8af409ce0daf, State: Initialized, Role: FOLLOWER
I20260806 21:06:54.811542 3629 raft_consensus.cc:348] T c020b47ed720452794e1e37ed8daf5db P 513daa018ebf4514894d5ef4d7cb795e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513daa018ebf4514894d5ef4d7cb795e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43373 } } peers { permanent_uuid: "3ae6436ae2ee446eacad8af409ce0daf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43009 } } peers { permanent_uuid: "130105c8e4b94acea85e45c8cbbfd8ad" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42173 } }
I20260806 21:06:54.811780 3630 consensus_queue.cc:260] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513daa018ebf4514894d5ef4d7cb795e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43373 } } peers { permanent_uuid: "3ae6436ae2ee446eacad8af409ce0daf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43009 } } peers { permanent_uuid: "130105c8e4b94acea85e45c8cbbfd8ad" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42173 } }
I20260806 21:06:54.811895 3629 raft_consensus.cc:374] T c020b47ed720452794e1e37ed8daf5db P 513daa018ebf4514894d5ef4d7cb795e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:54.811949 3629 raft_consensus.cc:729] T c020b47ed720452794e1e37ed8daf5db P 513daa018ebf4514894d5ef4d7cb795e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 513daa018ebf4514894d5ef4d7cb795e, State: Initialized, Role: FOLLOWER
I20260806 21:06:54.812433 3629 consensus_queue.cc:260] T c020b47ed720452794e1e37ed8daf5db P 513daa018ebf4514894d5ef4d7cb795e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513daa018ebf4514894d5ef4d7cb795e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43373 } } peers { permanent_uuid: "3ae6436ae2ee446eacad8af409ce0daf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43009 } } peers { permanent_uuid: "130105c8e4b94acea85e45c8cbbfd8ad" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42173 } }
I20260806 21:06:54.812573 3595 heartbeater.cc:503] Master 127.27.225.254:45179 was elected leader, sending a full tablet report...
I20260806 21:06:54.812983 3630 ts_tablet_manager.cc:1428] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260806 21:06:54.813458 3629 ts_tablet_manager.cc:1428] T c020b47ed720452794e1e37ed8daf5db P 513daa018ebf4514894d5ef4d7cb795e: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
W20260806 21:06:54.813680 3321 tablet.cc:2369] T c020b47ed720452794e1e37ed8daf5db P 513daa018ebf4514894d5ef4d7cb795e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:06:54.815340 3634 tablet_bootstrap.cc:492] T c020b47ed720452794e1e37ed8daf5db P 130105c8e4b94acea85e45c8cbbfd8ad: Bootstrap starting.
I20260806 21:06:54.816514 3319 heartbeater.cc:503] Master 127.27.225.254:45179 was elected leader, sending a full tablet report...
I20260806 21:06:54.818214 3634 tablet_bootstrap.cc:654] T c020b47ed720452794e1e37ed8daf5db P 130105c8e4b94acea85e45c8cbbfd8ad: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:54.819173 3634 log.cc:824] T c020b47ed720452794e1e37ed8daf5db P 130105c8e4b94acea85e45c8cbbfd8ad: Log is configured to *not* fsync() on all Append() calls
I20260806 21:06:54.822026 3634 tablet_bootstrap.cc:492] T c020b47ed720452794e1e37ed8daf5db P 130105c8e4b94acea85e45c8cbbfd8ad: No bootstrap required, opened a new log
I20260806 21:06:54.822264 3634 ts_tablet_manager.cc:1397] T c020b47ed720452794e1e37ed8daf5db P 130105c8e4b94acea85e45c8cbbfd8ad: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 21:06:54.826337 3634 raft_consensus.cc:348] T c020b47ed720452794e1e37ed8daf5db P 130105c8e4b94acea85e45c8cbbfd8ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513daa018ebf4514894d5ef4d7cb795e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43373 } } peers { permanent_uuid: "3ae6436ae2ee446eacad8af409ce0daf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43009 } } peers { permanent_uuid: "130105c8e4b94acea85e45c8cbbfd8ad" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42173 } }
I20260806 21:06:54.826591 3634 raft_consensus.cc:374] T c020b47ed720452794e1e37ed8daf5db P 130105c8e4b94acea85e45c8cbbfd8ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:54.826632 3634 raft_consensus.cc:729] T c020b47ed720452794e1e37ed8daf5db P 130105c8e4b94acea85e45c8cbbfd8ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130105c8e4b94acea85e45c8cbbfd8ad, State: Initialized, Role: FOLLOWER
I20260806 21:06:54.827118 3634 consensus_queue.cc:260] T c020b47ed720452794e1e37ed8daf5db P 130105c8e4b94acea85e45c8cbbfd8ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513daa018ebf4514894d5ef4d7cb795e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43373 } } peers { permanent_uuid: "3ae6436ae2ee446eacad8af409ce0daf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43009 } } peers { permanent_uuid: "130105c8e4b94acea85e45c8cbbfd8ad" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42173 } }
I20260806 21:06:54.827931 3458 heartbeater.cc:503] Master 127.27.225.254:45179 was elected leader, sending a full tablet report...
I20260806 21:06:54.828233 3634 ts_tablet_manager.cc:1428] T c020b47ed720452794e1e37ed8daf5db P 130105c8e4b94acea85e45c8cbbfd8ad: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 21:06:54.976863 3635 raft_consensus.cc:482] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:06:54.977121 3635 raft_consensus.cc:504] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513daa018ebf4514894d5ef4d7cb795e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43373 } } peers { permanent_uuid: "3ae6436ae2ee446eacad8af409ce0daf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43009 } } peers { permanent_uuid: "130105c8e4b94acea85e45c8cbbfd8ad" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42173 } }
I20260806 21:06:54.978555 3635 leader_election.cc:288] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 513daa018ebf4514894d5ef4d7cb795e (127.27.225.193:43373), 130105c8e4b94acea85e45c8cbbfd8ad (127.27.225.194:42173)
W20260806 21:06:54.978780 3597 tablet.cc:2369] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:06:54.982409 3628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.979195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:33163 (local address 127.27.225.194:42173)
0806 21:06:54.979323 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.979328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.979345 (+ 17us) server_negotiation.cc:407] Connection header received
0806 21:06:54.979408 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.979410 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.979453 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.979549 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:54.979849 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.980378 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.981364 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.981757 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.981936 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.982016 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.982183 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.982279 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 21:06:54.982416 3599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.978865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:42173 (local address 127.27.225.195:33163)
0806 21:06:54.979137 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:54.979161 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:54.979214 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:54.979572 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:54.979575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:54.979593 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:54.979733 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.979738 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.980502 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.980505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.981240 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.981248 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.981946 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.981949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.982052 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.982076 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 21:06:54.982168 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 21:06:54.982264 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":22}
I20260806 21:06:54.982491 3624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.978916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:34955 (local address 127.27.225.193:43373)
0806 21:06:54.979034 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:54.979038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:54.979290 (+ 252us) server_negotiation.cc:407] Connection header received
0806 21:06:54.979357 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:54.979360 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:54.979406 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:54.979484 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:54.979793 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.980305 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.981365 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.981774 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.981956 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.982008 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:54.982286 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 21:06:54.982367 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 21:06:54.982416 3600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:54.978777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:43373 (local address 127.27.225.195:34955)
0806 21:06:54.979062 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:54.979079 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:54.979193 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:54.979508 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:54.979511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:54.979527 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:54.979672 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.979679 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.980418 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.980424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.981232 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:54.981241 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:54.982054 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:54.982057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:54.982155 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:54.982182 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 21:06:54.982249 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 21:06:54.982304 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":3}
I20260806 21:06:54.983229 3410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c020b47ed720452794e1e37ed8daf5db" candidate_uuid: "3ae6436ae2ee446eacad8af409ce0daf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130105c8e4b94acea85e45c8cbbfd8ad" is_pre_election: true
I20260806 21:06:54.983219 3270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c020b47ed720452794e1e37ed8daf5db" candidate_uuid: "3ae6436ae2ee446eacad8af409ce0daf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "513daa018ebf4514894d5ef4d7cb795e" is_pre_election: true
I20260806 21:06:54.983573 3410 raft_consensus.cc:2454] T c020b47ed720452794e1e37ed8daf5db P 130105c8e4b94acea85e45c8cbbfd8ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae6436ae2ee446eacad8af409ce0daf in term 0.
I20260806 21:06:54.983574 3270 raft_consensus.cc:2454] T c020b47ed720452794e1e37ed8daf5db P 513daa018ebf4514894d5ef4d7cb795e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae6436ae2ee446eacad8af409ce0daf in term 0.
I20260806 21:06:54.984058 3481 leader_election.cc:302] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae6436ae2ee446eacad8af409ce0daf, 513daa018ebf4514894d5ef4d7cb795e; no voters:
I20260806 21:06:54.984404 3635 raft_consensus.cc:2781] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:06:54.984508 3635 raft_consensus.cc:482] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:06:54.984570 3635 raft_consensus.cc:3037] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:54.985882 3635 raft_consensus.cc:504] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513daa018ebf4514894d5ef4d7cb795e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43373 } } peers { permanent_uuid: "3ae6436ae2ee446eacad8af409ce0daf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43009 } } peers { permanent_uuid: "130105c8e4b94acea85e45c8cbbfd8ad" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42173 } }
I20260806 21:06:54.986274 3635 leader_election.cc:288] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf [CANDIDATE]: Term 1 election: Requested vote from peers 513daa018ebf4514894d5ef4d7cb795e (127.27.225.193:43373), 130105c8e4b94acea85e45c8cbbfd8ad (127.27.225.194:42173)
I20260806 21:06:54.986661 3270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c020b47ed720452794e1e37ed8daf5db" candidate_uuid: "3ae6436ae2ee446eacad8af409ce0daf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "513daa018ebf4514894d5ef4d7cb795e"
I20260806 21:06:54.986789 3410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c020b47ed720452794e1e37ed8daf5db" candidate_uuid: "3ae6436ae2ee446eacad8af409ce0daf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130105c8e4b94acea85e45c8cbbfd8ad"
I20260806 21:06:54.986817 3270 raft_consensus.cc:3037] T c020b47ed720452794e1e37ed8daf5db P 513daa018ebf4514894d5ef4d7cb795e [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:54.986963 3410 raft_consensus.cc:3037] T c020b47ed720452794e1e37ed8daf5db P 130105c8e4b94acea85e45c8cbbfd8ad [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:54.987927 3270 raft_consensus.cc:2454] T c020b47ed720452794e1e37ed8daf5db P 513daa018ebf4514894d5ef4d7cb795e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae6436ae2ee446eacad8af409ce0daf in term 1.
I20260806 21:06:54.987985 3410 raft_consensus.cc:2454] T c020b47ed720452794e1e37ed8daf5db P 130105c8e4b94acea85e45c8cbbfd8ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae6436ae2ee446eacad8af409ce0daf in term 1.
I20260806 21:06:54.988286 3481 leader_election.cc:302] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae6436ae2ee446eacad8af409ce0daf, 513daa018ebf4514894d5ef4d7cb795e; no voters:
I20260806 21:06:54.988478 3635 raft_consensus.cc:2781] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:54.988715 3635 raft_consensus.cc:686] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf [term 1 LEADER]: Becoming Leader. State: Replica: 3ae6436ae2ee446eacad8af409ce0daf, State: Running, Role: LEADER
I20260806 21:06:54.989082 3635 consensus_queue.cc:237] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513daa018ebf4514894d5ef4d7cb795e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43373 } } peers { permanent_uuid: "3ae6436ae2ee446eacad8af409ce0daf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43009 } } peers { permanent_uuid: "130105c8e4b94acea85e45c8cbbfd8ad" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42173 } }
I20260806 21:06:54.992651 2338 catalog_manager.cc:5495] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae6436ae2ee446eacad8af409ce0daf (127.27.225.195). New cstate: current_term: 1 leader_uuid: "3ae6436ae2ee446eacad8af409ce0daf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513daa018ebf4514894d5ef4d7cb795e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae6436ae2ee446eacad8af409ce0daf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130105c8e4b94acea85e45c8cbbfd8ad" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42173 } health_report { overall_health: UNKNOWN } } }
W20260806 21:06:55.025758 3460 tablet.cc:2369] T c020b47ed720452794e1e37ed8daf5db P 130105c8e4b94acea85e45c8cbbfd8ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:06:55.038820 3626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:55.034978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51478 (local address 127.27.225.195:43009)
0806 21:06:55.035112 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:55.035117 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:55.035133 (+ 16us) server_negotiation.cc:407] Connection header received
0806 21:06:55.035265 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:55.035267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:55.035323 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:55.035445 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:06:55.035695 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:55.036377 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:55.037251 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:55.037482 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:55.037589 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:55.037949 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:06:55.038396 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:06:55.038538 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:55.038602 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 21:06:55.038664 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 21:06:55.044974 3270 raft_consensus.cc:1261] T c020b47ed720452794e1e37ed8daf5db P 513daa018ebf4514894d5ef4d7cb795e [term 1 FOLLOWER]: Refusing update from remote peer 3ae6436ae2ee446eacad8af409ce0daf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:06:55.045022 3410 raft_consensus.cc:1261] T c020b47ed720452794e1e37ed8daf5db P 130105c8e4b94acea85e45c8cbbfd8ad [term 1 FOLLOWER]: Refusing update from remote peer 3ae6436ae2ee446eacad8af409ce0daf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:06:55.045796 3639 consensus_queue.cc:1035] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf [LEADER]: Connected to new peer: Peer: permanent_uuid: "130105c8e4b94acea85e45c8cbbfd8ad" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:06:55.046078 3635 consensus_queue.cc:1035] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf [LEADER]: Connected to new peer: Peer: permanent_uuid: "513daa018ebf4514894d5ef4d7cb795e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:06:55.053961 3642 mvcc.cc:205] Tried to move back new op lower bound from 7315662500016529408 to 7315662499798077440. Current Snapshot: MvccSnapshot[applied={T|T < 7315662500016529408}]
I20260806 21:06:55.058563 3643 mvcc.cc:205] Tried to move back new op lower bound from 7315662500016529408 to 7315662499798077440. Current Snapshot: MvccSnapshot[applied={T|T < 7315662500016529408}]
I20260806 21:06:55.059052 3644 mvcc.cc:205] Tried to move back new op lower bound from 7315662500016529408 to 7315662499798077440. Current Snapshot: MvccSnapshot[applied={T|T < 7315662500016529408}]
I20260806 21:06:55.067015 2338 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:06:55.067189 2338 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:06:55.071821 2338 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00de4fec8340498da672d706bcc37acf: Sending DeleteTablet for 3 replicas of tablet c020b47ed720452794e1e37ed8daf5db
I20260806 21:06:55.072664 3250 tablet_service.cc:1514] Processing DeleteTablet for tablet c020b47ed720452794e1e37ed8daf5db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:55 UTC) from {username='oryx', principal='oryx/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:54298
I20260806 21:06:55.072821 3390 tablet_service.cc:1514] Processing DeleteTablet for tablet c020b47ed720452794e1e37ed8daf5db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:55 UTC) from {username='oryx', principal='oryx/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:34014
I20260806 21:06:55.072837 3528 tablet_service.cc:1514] Processing DeleteTablet for tablet c020b47ed720452794e1e37ed8daf5db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:55 UTC) from {username='oryx', principal='oryx/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:51474
I20260806 21:06:55.073287 3653 tablet_replica.cc:330] T c020b47ed720452794e1e37ed8daf5db P 130105c8e4b94acea85e45c8cbbfd8ad: stopping tablet replica
I20260806 21:06:55.074016 3653 raft_consensus.cc:2229] T c020b47ed720452794e1e37ed8daf5db P 130105c8e4b94acea85e45c8cbbfd8ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:06:55.074395 3653 raft_consensus.cc:2258] T c020b47ed720452794e1e37ed8daf5db P 130105c8e4b94acea85e45c8cbbfd8ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:55.075309 3652 tablet_replica.cc:330] T c020b47ed720452794e1e37ed8daf5db P 513daa018ebf4514894d5ef4d7cb795e: stopping tablet replica
I20260806 21:06:55.075726 3652 raft_consensus.cc:2229] T c020b47ed720452794e1e37ed8daf5db P 513daa018ebf4514894d5ef4d7cb795e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:06:55.076138 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 2306
I20260806 21:06:55.076133 3652 raft_consensus.cc:2258] T c020b47ed720452794e1e37ed8daf5db P 513daa018ebf4514894d5ef4d7cb795e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:55.079047 3652 ts_tablet_manager.cc:1905] T c020b47ed720452794e1e37ed8daf5db P 513daa018ebf4514894d5ef4d7cb795e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:55.079825 3653 ts_tablet_manager.cc:1905] T c020b47ed720452794e1e37ed8daf5db P 130105c8e4b94acea85e45c8cbbfd8ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:55.082755 3652 ts_tablet_manager.cc:1918] T c020b47ed720452794e1e37ed8daf5db P 513daa018ebf4514894d5ef4d7cb795e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:06:55.082955 3652 log.cc:1196] T c020b47ed720452794e1e37ed8daf5db P 513daa018ebf4514894d5ef4d7cb795e: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-0/wal/wals/c020b47ed720452794e1e37ed8daf5db
I20260806 21:06:55.083411 3652 ts_tablet_manager.cc:1939] T c020b47ed720452794e1e37ed8daf5db P 513daa018ebf4514894d5ef4d7cb795e: Deleting consensus metadata
I20260806 21:06:55.085216 3654 tablet_replica.cc:330] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf: stopping tablet replica
I20260806 21:06:55.085717 3654 raft_consensus.cc:2229] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:06:55.086273 3653 ts_tablet_manager.cc:1918] T c020b47ed720452794e1e37ed8daf5db P 130105c8e4b94acea85e45c8cbbfd8ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:06:55.086401 3653 log.cc:1196] T c020b47ed720452794e1e37ed8daf5db P 130105c8e4b94acea85e45c8cbbfd8ad: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/wal/wals/c020b47ed720452794e1e37ed8daf5db
I20260806 21:06:55.086408 3654 raft_consensus.cc:2258] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:55.086902 3653 ts_tablet_manager.cc:1939] T c020b47ed720452794e1e37ed8daf5db P 130105c8e4b94acea85e45c8cbbfd8ad: Deleting consensus metadata
W20260806 21:06:55.087414 3203 connection.cc:535] server connection from 127.0.0.1:54298 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 21:06:55.090018 3654 ts_tablet_manager.cc:1905] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 21:06:55.092290 3342 connection.cc:441] server connection from 127.0.0.1:34014 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:34014 (request call id 1) could send its response
I20260806 21:06:55.094069 3654 ts_tablet_manager.cc:1918] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:06:55.094204 3654 log.cc:1196] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/c020b47ed720452794e1e37ed8daf5db
I20260806 21:06:55.095175 3654 ts_tablet_manager.cc:1939] T c020b47ed720452794e1e37ed8daf5db P 3ae6436ae2ee446eacad8af409ce0daf: Deleting consensus metadata
W20260806 21:06:55.096563 3482 connection.cc:441] server connection from 127.0.0.1:51474 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51474 (request call id 1) could send its response
I20260806 21:06:55.104610 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:55.074551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49330 (local address 127.27.225.253:34129)
0806 21:06:55.074677 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:55.074683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:55.075105 (+ 422us) server_negotiation.cc:407] Connection header received
0806 21:06:55.075310 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:55.075313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:55.075361 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:55.075411 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:55.077586 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:55.078698 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:55.091641 (+ 12943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:55.092036 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:55.092128 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:55.100299 (+ 8171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:55.100305 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:55.100311 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:55.100346 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:55.102605 (+ 2259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:55.103130 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:55.103134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:55.103136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:55.103243 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:55.103554 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:55.103558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:55.103559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:55.103797 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:55.103911 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:55.104289 (+ 378us) server_negotiation.cc:299] Negotiation successful
0806 21:06:55.104430 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 21:06:55.104902 3325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:55.091700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55110 (local address 127.27.225.252:34577)
0806 21:06:55.091855 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:55.091860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:55.093945 (+ 2085us) server_negotiation.cc:407] Connection header received
0806 21:06:55.094319 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:55.094323 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:55.094385 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:55.094452 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:55.095997 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:55.096834 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:55.098296 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:55.098659 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:55.098785 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:55.100369 (+ 1584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:55.100372 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:55.100377 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:55.100407 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:55.102605 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:55.103324 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:55.103330 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:55.103332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:55.103435 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:55.103820 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:55.103824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:55.103826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:55.104058 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:55.104179 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:55.104570 (+ 391us) server_negotiation.cc:299] Negotiation successful
0806 21:06:55.104698 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 21:06:55.106431 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:55.092817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49336 (local address 127.27.225.253:34129)
0806 21:06:55.092932 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:55.092937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:55.094550 (+ 1613us) server_negotiation.cc:407] Connection header received
0806 21:06:55.094820 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:55.094823 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:55.094879 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:55.094966 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:55.096826 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:55.097506 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:55.098533 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:55.099130 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:55.099484 (+ 354us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:55.101648 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:55.101656 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:55.101664 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:55.101703 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:55.104355 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:55.104834 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:55.104838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:55.104840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:55.104924 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:55.105266 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:55.105272 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:55.105276 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:55.105519 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:55.105615 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:55.106119 (+ 504us) server_negotiation.cc:299] Negotiation successful
0806 21:06:55.106266 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 21:06:55.115468 3608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:55.074943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55102 (local address 127.27.225.252:34577)
0806 21:06:55.075059 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:55.075063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:55.106922 (+ 31859us) server_negotiation.cc:407] Connection header received
0806 21:06:55.107032 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:55.107036 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:55.107089 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:55.107172 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:06:55.108065 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:55.108608 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:55.109670 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:55.110046 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:55.110119 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:55.111715 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:06:55.111719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:06:55.111722 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:06:55.111748 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:06:55.113484 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:55.114204 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:55.114208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:55.114210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:55.114297 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:06:55.114601 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:06:55.114605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:06:55.114607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:06:55.114856 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:06:55.114972 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:55.115238 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 21:06:55.115351 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 21:06:56.058441 3663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:56.057864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.194:41239)
0806 21:06:56.058188 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:56.058299 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":109,"threads_started":1}
I20260806 21:06:56.059605 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:56.059097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.195:51713)
0806 21:06:56.059390 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:56.059487 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":71,"threads_started":1}
W20260806 21:06:56.059052 3458 heartbeater.cc:644] Failed to heartbeat to 127.27.225.254:45179 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
W20260806 21:06:56.060263 3595 heartbeater.cc:644] Failed to heartbeat to 127.27.225.254:45179 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
I20260806 21:06:56.060781 3665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:56.060134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.193:33247)
0806 21:06:56.060565 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:56.060666 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":72,"threads_started":1}
W20260806 21:06:56.061120 3319 heartbeater.cc:644] Failed to heartbeat to 127.27.225.254:45179 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
I20260806 21:06:56.572934 3666 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00de4fec8340498da672d706bcc37acf)
I20260806 21:06:56.573272 3666 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: false } }
I20260806 21:06:56.574092 3667 raft_consensus.cc:482] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00de4fec8340498da672d706bcc37acf)
I20260806 21:06:56.574297 3667 raft_consensus.cc:504] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: false } }
I20260806 21:06:56.574652 3666 leader_election.cc:288] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 00de4fec8340498da672d706bcc37acf (127.27.225.254:45179), db766e7b4c4b4ae1ba460ea4cf5a6131 (127.27.225.252:34577)
I20260806 21:06:56.575659 3668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:56.574776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.0.0.1:52992)
0806 21:06:56.575481 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:56.575559 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":60,"threads_started":1}
I20260806 21:06:56.575790 3667 leader_election.cc:288] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 00de4fec8340498da672d706bcc37acf (127.27.225.254:45179), 4859fba6ee99437aa8a0285c7270fe12 (127.27.225.253:34129)
W20260806 21:06:56.576123 2934 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
I20260806 21:06:56.576459 3672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:56.575898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.0.0.1:53004)
0806 21:06:56.576293 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:56.576374 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":58,"threads_started":1}
W20260806 21:06:56.576689 3122 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
I20260806 21:06:56.577512 3668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:56.577291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.0.0.1:53014)
0806 21:06:56.577395 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:56.577444 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 21:06:56.577689 3671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:56.577441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.0.0.1:53030)
0806 21:06:56.577550 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:56.577607 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 21:06:56.577682 2934 leader_election.cc:334] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 00de4fec8340498da672d706bcc37acf (127.27.225.254:45179): Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
W20260806 21:06:56.577790 3122 leader_election.cc:334] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 00de4fec8340498da672d706bcc37acf (127.27.225.254:45179): Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
I20260806 21:06:56.580518 3670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:56.575261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55122 (local address 127.27.225.252:34577)
0806 21:06:56.575959 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:56.575964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:56.576672 (+ 708us) server_negotiation.cc:407] Connection header received
0806 21:06:56.576721 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:56.576724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:56.576834 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:56.576920 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:56.577589 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:56.578288 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:56.579475 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:56.579885 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:56.580015 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:56.580070 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:56.580315 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 21:06:56.580389 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":77,"threads_started":1}
I20260806 21:06:56.580513 3669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:56.575118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:34577 (local address 127.0.0.1:55122)
0806 21:06:56.576356 (+ 1238us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:56.576373 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:56.576511 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:56.577094 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:56.577097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:56.577111 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:56.577225 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:56.577231 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:56.578405 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:56.578408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:56.579343 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:56.579351 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:56.580010 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:56.580014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:56.580179 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:56.580197 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 21:06:56.580297 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 21:06:56.580382 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1158,"mutex_wait_us":49,"thread_start_us":74,"threads_started":1}
I20260806 21:06:56.580724 3672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:56.575977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:34129 (local address 127.0.0.1:49346)
0806 21:06:56.576609 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:56.576623 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:56.576717 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:56.577095 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:56.577098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:56.577111 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:56.577472 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:56.577480 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:56.578814 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:56.578816 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:56.579592 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:56.579601 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:56.580300 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:56.580303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:56.580394 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:56.580416 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 21:06:56.580516 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 21:06:56.580614 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":52,"threads_started":1}
I20260806 21:06:56.580725 3674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:56.576487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49346 (local address 127.27.225.253:34129)
0806 21:06:56.576871 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:56.576882 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:56.576898 (+ 16us) server_negotiation.cc:407] Connection header received
0806 21:06:56.576939 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:56.576942 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:56.576979 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:56.577893 (+ 914us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:56.577996 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:56.578694 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:56.579725 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:56.580178 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:56.580307 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:56.580354 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:56.580531 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 21:06:56.580605 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":316,"thread_start_us":107,"threads_started":1}
I20260806 21:06:56.581332 2957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4859fba6ee99437aa8a0285c7270fe12" is_pre_election: true
I20260806 21:06:56.581490 3144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4859fba6ee99437aa8a0285c7270fe12" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" is_pre_election: true
I20260806 21:06:56.581538 2957 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db766e7b4c4b4ae1ba460ea4cf5a6131 in term 1.
I20260806 21:06:56.581691 3144 raft_consensus.cc:2454] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4859fba6ee99437aa8a0285c7270fe12 in term 1.
I20260806 21:06:56.581924 3120 leader_election.cc:302] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4859fba6ee99437aa8a0285c7270fe12, db766e7b4c4b4ae1ba460ea4cf5a6131; no voters: 00de4fec8340498da672d706bcc37acf
I20260806 21:06:56.582078 2934 leader_election.cc:302] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4859fba6ee99437aa8a0285c7270fe12, db766e7b4c4b4ae1ba460ea4cf5a6131; no voters: 00de4fec8340498da672d706bcc37acf
I20260806 21:06:56.582208 3667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 21:06:56.582274 3667 raft_consensus.cc:482] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 00de4fec8340498da672d706bcc37acf)
I20260806 21:06:56.582327 3667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [term 1 FOLLOWER]: Advancing to term 2
I20260806 21:06:56.582370 3666 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 21:06:56.582433 3666 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 00de4fec8340498da672d706bcc37acf)
I20260806 21:06:56.582482 3666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 1 FOLLOWER]: Advancing to term 2
I20260806 21:06:56.583406 3667 raft_consensus.cc:504] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: false } }
I20260806 21:06:56.583452 3666 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: false } }
I20260806 21:06:56.583807 3667 leader_election.cc:288] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [CANDIDATE]: Term 2 election: Requested vote from peers 00de4fec8340498da672d706bcc37acf (127.27.225.254:45179), 4859fba6ee99437aa8a0285c7270fe12 (127.27.225.253:34129)
I20260806 21:06:56.583861 3666 leader_election.cc:288] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [CANDIDATE]: Term 2 election: Requested vote from peers 00de4fec8340498da672d706bcc37acf (127.27.225.254:45179), db766e7b4c4b4ae1ba460ea4cf5a6131 (127.27.225.252:34577)
I20260806 21:06:56.584340 3669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:56.584066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.0.0.1:53042)
0806 21:06:56.584189 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:56.584269 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 21:06:56.584408 2957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4859fba6ee99437aa8a0285c7270fe12"
I20260806 21:06:56.584420 3144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4859fba6ee99437aa8a0285c7270fe12" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131"
I20260806 21:06:56.584581 2957 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate db766e7b4c4b4ae1ba460ea4cf5a6131 in current term 2: Already voted for candidate 4859fba6ee99437aa8a0285c7270fe12 in this term.
I20260806 21:06:56.584587 3144 raft_consensus.cc:2379] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4859fba6ee99437aa8a0285c7270fe12 in current term 2: Already voted for candidate db766e7b4c4b4ae1ba460ea4cf5a6131 in this term.
I20260806 21:06:56.585340 3669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:56.585140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.0.0.1:53048)
0806 21:06:56.585239 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:56.585289 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 21:06:56.585407 3672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:56.583929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.0.0.1:53034)
0806 21:06:56.585278 (+ 1349us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:56.585322 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1291}
W20260806 21:06:56.585448 2934 leader_election.cc:334] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 00de4fec8340498da672d706bcc37acf (127.27.225.254:45179): Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
I20260806 21:06:56.585516 2934 leader_election.cc:302] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4859fba6ee99437aa8a0285c7270fe12; no voters: 00de4fec8340498da672d706bcc37acf, db766e7b4c4b4ae1ba460ea4cf5a6131
I20260806 21:06:56.585656 3666 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 21:06:56.586041 3672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:56.585846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.0.0.1:53056)
0806 21:06:56.585927 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:56.585979 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 21:06:56.586139 3122 leader_election.cc:334] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 00de4fec8340498da672d706bcc37acf (127.27.225.254:45179): Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
I20260806 21:06:56.586201 3122 leader_election.cc:302] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db766e7b4c4b4ae1ba460ea4cf5a6131; no voters: 00de4fec8340498da672d706bcc37acf, 4859fba6ee99437aa8a0285c7270fe12
I20260806 21:06:56.586321 3667 raft_consensus.cc:2726] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 21:06:57.061318 3677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:57.060868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.195:55263)
0806 21:06:57.061124 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:57.061192 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":105,"threads_started":1}
I20260806 21:06:57.061338 3676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:57.060855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.194:34187)
0806 21:06:57.061118 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:57.061186 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":101,"threads_started":1}
I20260806 21:06:57.062098 3678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:57.061642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.193:54433)
0806 21:06:57.061961 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:57.062021 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":93,"threads_started":1}
I20260806 21:06:58.063176 3680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.062528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.194:52305)
0806 21:06:58.062911 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.063015 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":71,"threads_started":1}
I20260806 21:06:58.063511 3681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.062929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.193:56773)
0806 21:06:58.063264 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.063359 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":96,"threads_started":1}
I20260806 21:06:58.064153 3679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.062383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.195:40133)
0806 21:06:58.063976 (+ 1593us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.064056 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1515,"thread_start_us":100,"threads_started":1}
I20260806 21:06:58.509768 3682 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:06:58.509956 3682 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: false } }
I20260806 21:06:58.510468 3682 leader_election.cc:288] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00de4fec8340498da672d706bcc37acf (127.27.225.254:45179), db766e7b4c4b4ae1ba460ea4cf5a6131 (127.27.225.252:34577)
I20260806 21:06:58.511114 3683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.510701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.0.0.1:53330)
0806 21:06:58.510916 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.510996 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
I20260806 21:06:58.511426 3144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4859fba6ee99437aa8a0285c7270fe12" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" is_pre_election: true
I20260806 21:06:58.511641 3144 raft_consensus.cc:2454] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4859fba6ee99437aa8a0285c7270fe12 in term 2.
I20260806 21:06:58.511988 3683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.511724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.0.0.1:53334)
0806 21:06:58.511857 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.511917 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 21:06:58.511945 2934 leader_election.cc:302] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4859fba6ee99437aa8a0285c7270fe12, db766e7b4c4b4ae1ba460ea4cf5a6131; no voters:
I20260806 21:06:58.512229 3682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 21:06:58.512316 3682 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260806 21:06:58.512229 2934 leader_election.cc:334] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 00de4fec8340498da672d706bcc37acf (127.27.225.254:45179): Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
I20260806 21:06:58.512356 3682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 2 FOLLOWER]: Advancing to term 3
I20260806 21:06:58.513602 3682 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: false } }
I20260806 21:06:58.514034 3682 leader_election.cc:288] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [CANDIDATE]: Term 3 election: Requested vote from peers 00de4fec8340498da672d706bcc37acf (127.27.225.254:45179), db766e7b4c4b4ae1ba460ea4cf5a6131 (127.27.225.252:34577)
I20260806 21:06:58.514381 3683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.514148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.0.0.1:53346)
0806 21:06:58.514240 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.514298 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 21:06:58.514631 3144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4859fba6ee99437aa8a0285c7270fe12" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131"
I20260806 21:06:58.514818 3144 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [term 2 FOLLOWER]: Advancing to term 3
I20260806 21:06:58.515184 3683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.514938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.0.0.1:53354)
0806 21:06:58.515069 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.515121 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 21:06:58.515290 2934 leader_election.cc:334] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 00de4fec8340498da672d706bcc37acf (127.27.225.254:45179): Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
I20260806 21:06:58.516131 3144 raft_consensus.cc:2454] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4859fba6ee99437aa8a0285c7270fe12 in term 3.
I20260806 21:06:58.516530 2934 leader_election.cc:302] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4859fba6ee99437aa8a0285c7270fe12, db766e7b4c4b4ae1ba460ea4cf5a6131; no voters: 00de4fec8340498da672d706bcc37acf
I20260806 21:06:58.516703 3682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 21:06:58.516899 3682 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [term 3 LEADER]: Becoming Leader. State: Replica: 4859fba6ee99437aa8a0285c7270fe12, State: Running, Role: LEADER
I20260806 21:06:58.517216 3682 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: false } }
I20260806 21:06:58.518428 3685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4859fba6ee99437aa8a0285c7270fe12. Latest consensus state: current_term: 3 leader_uuid: "4859fba6ee99437aa8a0285c7270fe12" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: false } } }
I20260806 21:06:58.518565 3685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:58.518924 3687 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:06:58.521551 3687 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:06:58.522087 3687 catalog_manager.cc:1259] Loaded cluster ID: 42ffcff42f5a4ac2a89ff0d847fddd0b
I20260806 21:06:58.522142 3687 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:06:58.522806 3687 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:06:58.523240 3687 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12: Loaded TSK: 0
I20260806 21:06:58.523998 3687 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:06:58.584264 2946 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49336:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:06:58.587630 2946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:06:58.590854 3683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.590568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.0.0.1:53366)
0806 21:06:58.590699 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.590769 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 21:06:58.590888 3144 raft_consensus.cc:1261] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [term 3 FOLLOWER]: Refusing update from remote peer 4859fba6ee99437aa8a0285c7270fe12: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260806 21:06:58.591394 3682 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260806 21:06:58.591791 3683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.591500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.0.0.1:53374)
0806 21:06:58.591647 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.591727 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260806 21:06:58.591989 2934 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 -> Peer 00de4fec8340498da672d706bcc37acf (127.27.225.254:45179): Couldn't send request to peer 00de4fec8340498da672d706bcc37acf. Status: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 21:06:58.593094 3689 sys_catalog.cc:455] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4859fba6ee99437aa8a0285c7270fe12. Latest consensus state: current_term: 3 leader_uuid: "4859fba6ee99437aa8a0285c7270fe12" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: false } } }
I20260806 21:06:58.593228 3689 sys_catalog.cc:458] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:06:58.594523 3682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "4859fba6ee99437aa8a0285c7270fe12" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: false } } }
I20260806 21:06:58.594661 3682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12 [sys.catalog]: This master's current role is: LEADER
I20260806 21:06:58.594841 3689 sys_catalog.cc:455] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "4859fba6ee99437aa8a0285c7270fe12" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00de4fec8340498da672d706bcc37acf" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45179 } } peers { permanent_uuid: "4859fba6ee99437aa8a0285c7270fe12" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 34129 } attrs { promote: false } } peers { permanent_uuid: "db766e7b4c4b4ae1ba460ea4cf5a6131" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 34577 } attrs { promote: false } } }
I20260806 21:06:58.594991 3689 sys_catalog.cc:458] T 00000000000000000000000000000000 P db766e7b4c4b4ae1ba460ea4cf5a6131 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:06:58.605717 3683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.600792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:43009 (local address 127.0.0.1:51488)
0806 21:06:58.600945 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.600961 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:58.601054 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:58.602084 (+ 1030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:58.602088 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:58.602107 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:58.602207 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:58.602212 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:58.603255 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:58.603260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:58.604074 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:58.604086 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:58.605272 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:58.605275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:58.605377 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:58.605407 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 21:06:58.605460 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 21:06:58.605542 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":1}
I20260806 21:06:58.606391 3695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.601136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:42173 (local address 127.0.0.1:34028)
0806 21:06:58.601508 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.601523 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:58.601611 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:58.602426 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:58.602429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:58.602445 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:58.602515 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:58.602519 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:58.603534 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:58.603538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:58.604743 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:58.604752 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:58.605623 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:58.605627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:58.605729 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:58.605756 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 21:06:58.606150 (+ 394us) client_negotiation.cc:240] Negotiation successful
0806 21:06:58.606235 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":24,"thread_start_us":145,"threads_started":1}
I20260806 21:06:58.606389 3696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.601249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34028 (local address 127.27.225.194:42173)
0806 21:06:58.602147 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:58.602152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:58.602166 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:06:58.602224 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:58.602228 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:58.602303 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:58.602394 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:58.602641 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:58.603397 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:58.604884 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:58.605501 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:58.605617 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:58.605669 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:58.606174 (+ 505us) server_negotiation.cc:299] Negotiation successful
0806 21:06:58.606265 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":818,"thread_start_us":81,"threads_started":1}
I20260806 21:06:58.606405 3699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.601535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51488 (local address 127.27.225.195:43009)
0806 21:06:58.601781 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:58.601785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:58.601800 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:06:58.601877 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:58.601880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:58.601937 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:58.602068 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:58.602319 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:58.603108 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:58.604206 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:58.605141 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:58.606104 (+ 963us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:58.606171 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:58.606207 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 21:06:58.606282 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":72,"threads_started":1}
I20260806 21:06:58.606535 3698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.601522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54302 (local address 127.27.225.193:43373)
0806 21:06:58.601769 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:58.601773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:58.602733 (+ 960us) server_negotiation.cc:407] Connection header received
0806 21:06:58.602841 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:58.602844 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:58.602895 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:58.602980 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:58.603225 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:58.603966 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:58.605222 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:58.605741 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:58.605915 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:58.605971 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:58.606316 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 21:06:58.606397 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":84,"threads_started":1}
I20260806 21:06:58.607192 3697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.601404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:43373 (local address 127.0.0.1:54302)
0806 21:06:58.602627 (+ 1223us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.602641 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:58.602747 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:58.603007 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:58.603009 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:58.603024 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:58.603108 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:58.603114 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:58.604092 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:58.604096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:58.605068 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:58.605077 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:58.605923 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:58.605928 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:58.606057 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:58.606080 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 21:06:58.606148 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 21:06:58.606223 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1130,"thread_start_us":88,"threads_started":1}
I20260806 21:06:58.607422 3528 tablet_service.cc:1467] Processing CreateTablet for tablet dc9962ef9b894b8fa8bfd4f9a268abd9 (DEFAULT_TABLE table=test-table [id=ec13d3a927594c79b6bcaa8c18debfa9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:58.607752 3528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc9962ef9b894b8fa8bfd4f9a268abd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:58.607801 3390 tablet_service.cc:1467] Processing CreateTablet for tablet dc9962ef9b894b8fa8bfd4f9a268abd9 (DEFAULT_TABLE table=test-table [id=ec13d3a927594c79b6bcaa8c18debfa9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:58.608080 3390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc9962ef9b894b8fa8bfd4f9a268abd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:58.610090 3700 tablet_bootstrap.cc:492] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 3ae6436ae2ee446eacad8af409ce0daf: Bootstrap starting.
I20260806 21:06:58.610378 3701 tablet_bootstrap.cc:492] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 130105c8e4b94acea85e45c8cbbfd8ad: Bootstrap starting.
I20260806 21:06:58.610384 3250 tablet_service.cc:1467] Processing CreateTablet for tablet dc9962ef9b894b8fa8bfd4f9a268abd9 (DEFAULT_TABLE table=test-table [id=ec13d3a927594c79b6bcaa8c18debfa9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:06:58.610668 3250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc9962ef9b894b8fa8bfd4f9a268abd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:06:58.611104 3700 tablet_bootstrap.cc:654] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 3ae6436ae2ee446eacad8af409ce0daf: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:58.611418 3701 tablet_bootstrap.cc:654] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 130105c8e4b94acea85e45c8cbbfd8ad: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:58.612877 3701 tablet_bootstrap.cc:492] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 130105c8e4b94acea85e45c8cbbfd8ad: No bootstrap required, opened a new log
I20260806 21:06:58.613081 3701 ts_tablet_manager.cc:1397] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 130105c8e4b94acea85e45c8cbbfd8ad: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:06:58.613284 3704 tablet_bootstrap.cc:492] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 513daa018ebf4514894d5ef4d7cb795e: Bootstrap starting.
I20260806 21:06:58.613727 3701 raft_consensus.cc:348] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 130105c8e4b94acea85e45c8cbbfd8ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae6436ae2ee446eacad8af409ce0daf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43009 } } peers { permanent_uuid: "130105c8e4b94acea85e45c8cbbfd8ad" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42173 } } peers { permanent_uuid: "513daa018ebf4514894d5ef4d7cb795e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43373 } }
I20260806 21:06:58.613899 3701 raft_consensus.cc:374] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 130105c8e4b94acea85e45c8cbbfd8ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:58.613951 3701 raft_consensus.cc:729] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 130105c8e4b94acea85e45c8cbbfd8ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130105c8e4b94acea85e45c8cbbfd8ad, State: Initialized, Role: FOLLOWER
I20260806 21:06:58.614104 3701 consensus_queue.cc:260] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 130105c8e4b94acea85e45c8cbbfd8ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae6436ae2ee446eacad8af409ce0daf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43009 } } peers { permanent_uuid: "130105c8e4b94acea85e45c8cbbfd8ad" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42173 } } peers { permanent_uuid: "513daa018ebf4514894d5ef4d7cb795e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43373 } }
I20260806 21:06:58.614180 3704 tablet_bootstrap.cc:654] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 513daa018ebf4514894d5ef4d7cb795e: Neither blocks nor log segments found. Creating new log.
I20260806 21:06:58.614454 3700 tablet_bootstrap.cc:492] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 3ae6436ae2ee446eacad8af409ce0daf: No bootstrap required, opened a new log
I20260806 21:06:58.614461 3701 ts_tablet_manager.cc:1428] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 130105c8e4b94acea85e45c8cbbfd8ad: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:06:58.614557 3700 ts_tablet_manager.cc:1397] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 3ae6436ae2ee446eacad8af409ce0daf: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 21:06:58.614972 3700 raft_consensus.cc:348] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 3ae6436ae2ee446eacad8af409ce0daf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae6436ae2ee446eacad8af409ce0daf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43009 } } peers { permanent_uuid: "130105c8e4b94acea85e45c8cbbfd8ad" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42173 } } peers { permanent_uuid: "513daa018ebf4514894d5ef4d7cb795e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43373 } }
I20260806 21:06:58.615093 3700 raft_consensus.cc:374] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 3ae6436ae2ee446eacad8af409ce0daf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:58.615130 3700 raft_consensus.cc:729] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 3ae6436ae2ee446eacad8af409ce0daf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae6436ae2ee446eacad8af409ce0daf, State: Initialized, Role: FOLLOWER
I20260806 21:06:58.615274 3700 consensus_queue.cc:260] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 3ae6436ae2ee446eacad8af409ce0daf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae6436ae2ee446eacad8af409ce0daf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43009 } } peers { permanent_uuid: "130105c8e4b94acea85e45c8cbbfd8ad" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42173 } } peers { permanent_uuid: "513daa018ebf4514894d5ef4d7cb795e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43373 } }
I20260806 21:06:58.615559 3707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.615065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.194:48265)
0806 21:06:58.615378 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.615449 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":107,"threads_started":1}
I20260806 21:06:58.615633 3700 ts_tablet_manager.cc:1428] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 3ae6436ae2ee446eacad8af409ce0daf: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 21:06:58.617888 3707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.616888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.194:34625)
0806 21:06:58.617647 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.617702 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":679}
I20260806 21:06:58.618352 3596 heartbeater.cc:503] Master 127.27.225.253:34129 was elected leader, sending a full tablet report...
I20260806 21:06:58.618537 3709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.616295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.195:47547)
0806 21:06:58.618395 (+ 2100us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.618453 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2035,"thread_start_us":103,"threads_started":1}
I20260806 21:06:58.619165 3704 tablet_bootstrap.cc:492] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 513daa018ebf4514894d5ef4d7cb795e: No bootstrap required, opened a new log
I20260806 21:06:58.619170 3709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.618953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.195:60899)
0806 21:06:58.619064 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.619113 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 21:06:58.619274 3704 ts_tablet_manager.cc:1397] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 513daa018ebf4514894d5ef4d7cb795e: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260806 21:06:58.619679 3704 raft_consensus.cc:348] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 513daa018ebf4514894d5ef4d7cb795e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae6436ae2ee446eacad8af409ce0daf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43009 } } peers { permanent_uuid: "130105c8e4b94acea85e45c8cbbfd8ad" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42173 } } peers { permanent_uuid: "513daa018ebf4514894d5ef4d7cb795e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43373 } }
I20260806 21:06:58.619800 3704 raft_consensus.cc:374] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 513daa018ebf4514894d5ef4d7cb795e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:06:58.619844 3704 raft_consensus.cc:729] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 513daa018ebf4514894d5ef4d7cb795e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 513daa018ebf4514894d5ef4d7cb795e, State: Initialized, Role: FOLLOWER
I20260806 21:06:58.619982 3704 consensus_queue.cc:260] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 513daa018ebf4514894d5ef4d7cb795e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae6436ae2ee446eacad8af409ce0daf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43009 } } peers { permanent_uuid: "130105c8e4b94acea85e45c8cbbfd8ad" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42173 } } peers { permanent_uuid: "513daa018ebf4514894d5ef4d7cb795e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43373 } }
I20260806 21:06:58.620342 3704 ts_tablet_manager.cc:1428] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 513daa018ebf4514894d5ef4d7cb795e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:06:58.621078 3711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.620634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.193:46583)
0806 21:06:58.620892 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.620951 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":97,"threads_started":1}
I20260806 21:06:58.621687 3320 heartbeater.cc:503] Master 127.27.225.253:34129 was elected leader, sending a full tablet report...
I20260806 21:06:58.621778 3711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.621520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.193:37281)
0806 21:06:58.621613 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.621665 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 21:06:58.668498 3710 raft_consensus.cc:482] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 513daa018ebf4514894d5ef4d7cb795e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:06:58.668666 3710 raft_consensus.cc:504] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 513daa018ebf4514894d5ef4d7cb795e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae6436ae2ee446eacad8af409ce0daf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43009 } } peers { permanent_uuid: "130105c8e4b94acea85e45c8cbbfd8ad" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42173 } } peers { permanent_uuid: "513daa018ebf4514894d5ef4d7cb795e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43373 } }
I20260806 21:06:58.669811 3710 leader_election.cc:288] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 513daa018ebf4514894d5ef4d7cb795e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae6436ae2ee446eacad8af409ce0daf (127.27.225.195:43009), 130105c8e4b94acea85e45c8cbbfd8ad (127.27.225.194:42173)
I20260806 21:06:58.673887 3699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.670207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:45965 (local address 127.27.225.195:43009)
0806 21:06:58.670368 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:58.670372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:58.670396 (+ 24us) server_negotiation.cc:407] Connection header received
0806 21:06:58.670590 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:58.670593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:58.670650 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:58.670748 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:58.671046 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:58.671557 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:58.672729 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:58.673136 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:58.673272 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:58.673333 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:58.673629 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 21:06:58.673717 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 21:06:58.673887 3696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.670257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:48243 (local address 127.27.225.194:42173)
0806 21:06:58.670382 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:58.670388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:58.670683 (+ 295us) server_negotiation.cc:407] Connection header received
0806 21:06:58.670825 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:58.670828 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:58.670879 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:58.670966 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:06:58.671228 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:58.671739 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:58.672801 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:58.673216 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:58.673341 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:58.673397 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:58.673664 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 21:06:58.673745 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 21:06:58.673887 3711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.670115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:43009 (local address 127.27.225.193:45965)
0806 21:06:58.670323 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.670340 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:58.670449 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:58.670776 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:58.670779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:58.670799 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:58.670928 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:58.670933 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:58.671671 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:58.671678 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:58.672594 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:58.672604 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:58.673286 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:58.673290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:58.673417 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:58.673500 (+ 83us) client_negotiation.cc:769] Sending connection context
0806 21:06:58.673643 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 21:06:58.673719 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":99}
I20260806 21:06:58.673903 3712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.670137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:42173 (local address 127.27.225.193:48243)
0806 21:06:58.670568 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.670593 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 21:06:58.670708 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:06:58.670978 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:06:58.670981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:06:58.670996 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:06:58.671107 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:58.671113 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:58.671864 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:58.671867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:58.672672 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:06:58.672681 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:06:58.673337 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:06:58.673341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:06:58.673485 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:58.673502 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 21:06:58.673633 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 21:06:58.673718 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":21,"thread_start_us":140,"threads_started":1}
I20260806 21:06:58.674574 3548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc9962ef9b894b8fa8bfd4f9a268abd9" candidate_uuid: "513daa018ebf4514894d5ef4d7cb795e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae6436ae2ee446eacad8af409ce0daf" is_pre_election: true
I20260806 21:06:58.674649 3410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc9962ef9b894b8fa8bfd4f9a268abd9" candidate_uuid: "513daa018ebf4514894d5ef4d7cb795e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130105c8e4b94acea85e45c8cbbfd8ad" is_pre_election: true
I20260806 21:06:58.674839 3410 raft_consensus.cc:2454] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 130105c8e4b94acea85e45c8cbbfd8ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 513daa018ebf4514894d5ef4d7cb795e in term 0.
I20260806 21:06:58.674852 3548 raft_consensus.cc:2454] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 3ae6436ae2ee446eacad8af409ce0daf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 513daa018ebf4514894d5ef4d7cb795e in term 0.
I20260806 21:06:58.675238 3202 leader_election.cc:302] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 513daa018ebf4514894d5ef4d7cb795e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130105c8e4b94acea85e45c8cbbfd8ad, 513daa018ebf4514894d5ef4d7cb795e; no voters:
I20260806 21:06:58.675524 3710 raft_consensus.cc:2781] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 513daa018ebf4514894d5ef4d7cb795e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:06:58.675590 3710 raft_consensus.cc:482] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 513daa018ebf4514894d5ef4d7cb795e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:06:58.675617 3710 raft_consensus.cc:3037] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 513daa018ebf4514894d5ef4d7cb795e [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:58.676661 3710 raft_consensus.cc:504] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 513daa018ebf4514894d5ef4d7cb795e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae6436ae2ee446eacad8af409ce0daf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43009 } } peers { permanent_uuid: "130105c8e4b94acea85e45c8cbbfd8ad" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42173 } } peers { permanent_uuid: "513daa018ebf4514894d5ef4d7cb795e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43373 } }
I20260806 21:06:58.677055 3710 leader_election.cc:288] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 513daa018ebf4514894d5ef4d7cb795e [CANDIDATE]: Term 1 election: Requested vote from peers 3ae6436ae2ee446eacad8af409ce0daf (127.27.225.195:43009), 130105c8e4b94acea85e45c8cbbfd8ad (127.27.225.194:42173)
I20260806 21:06:58.677414 3410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc9962ef9b894b8fa8bfd4f9a268abd9" candidate_uuid: "513daa018ebf4514894d5ef4d7cb795e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130105c8e4b94acea85e45c8cbbfd8ad"
I20260806 21:06:58.677515 3410 raft_consensus.cc:3037] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 130105c8e4b94acea85e45c8cbbfd8ad [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:58.677452 3548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc9962ef9b894b8fa8bfd4f9a268abd9" candidate_uuid: "513daa018ebf4514894d5ef4d7cb795e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae6436ae2ee446eacad8af409ce0daf"
I20260806 21:06:58.677629 3548 raft_consensus.cc:3037] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 3ae6436ae2ee446eacad8af409ce0daf [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:06:58.678644 3410 raft_consensus.cc:2454] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 130105c8e4b94acea85e45c8cbbfd8ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 513daa018ebf4514894d5ef4d7cb795e in term 1.
I20260806 21:06:58.678998 3202 leader_election.cc:302] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 513daa018ebf4514894d5ef4d7cb795e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130105c8e4b94acea85e45c8cbbfd8ad, 513daa018ebf4514894d5ef4d7cb795e; no voters:
I20260806 21:06:58.679123 3548 raft_consensus.cc:2454] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 3ae6436ae2ee446eacad8af409ce0daf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 513daa018ebf4514894d5ef4d7cb795e in term 1.
I20260806 21:06:58.679252 3710 raft_consensus.cc:2781] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 513daa018ebf4514894d5ef4d7cb795e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:06:58.679502 3710 raft_consensus.cc:686] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 513daa018ebf4514894d5ef4d7cb795e [term 1 LEADER]: Becoming Leader. State: Replica: 513daa018ebf4514894d5ef4d7cb795e, State: Running, Role: LEADER
I20260806 21:06:58.679951 3710 consensus_queue.cc:237] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 513daa018ebf4514894d5ef4d7cb795e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae6436ae2ee446eacad8af409ce0daf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43009 } } peers { permanent_uuid: "130105c8e4b94acea85e45c8cbbfd8ad" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42173 } } peers { permanent_uuid: "513daa018ebf4514894d5ef4d7cb795e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43373 } }
I20260806 21:06:58.680298 3712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.680014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.193:57437)
0806 21:06:58.680163 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.680229 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 21:06:58.682456 2946 catalog_manager.cc:5495] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 513daa018ebf4514894d5ef4d7cb795e reported cstate change: term changed from 0 to 1, leader changed from <none> to 513daa018ebf4514894d5ef4d7cb795e (127.27.225.193). New cstate: current_term: 1 leader_uuid: "513daa018ebf4514894d5ef4d7cb795e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae6436ae2ee446eacad8af409ce0daf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130105c8e4b94acea85e45c8cbbfd8ad" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "513daa018ebf4514894d5ef4d7cb795e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43373 } health_report { overall_health: HEALTHY } } }
I20260806 21:06:58.708375 3698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.704680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54312 (local address 127.27.225.193:43373)
0806 21:06:58.704840 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 21:06:58.704845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:06:58.704862 (+ 17us) server_negotiation.cc:407] Connection header received
0806 21:06:58.705020 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:06:58.705022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:06:58.705079 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:06:58.705208 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:06:58.705470 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:58.706027 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:58.706895 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:06:58.707123 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:06:58.707233 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:06:58.707511 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:06:58.707965 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:06:58.708093 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 21:06:58.708166 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 21:06:58.708225 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 21:06:58.711483 3410 raft_consensus.cc:1261] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 130105c8e4b94acea85e45c8cbbfd8ad [term 1 FOLLOWER]: Refusing update from remote peer 513daa018ebf4514894d5ef4d7cb795e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:06:58.711679 3548 raft_consensus.cc:1261] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 3ae6436ae2ee446eacad8af409ce0daf [term 1 FOLLOWER]: Refusing update from remote peer 513daa018ebf4514894d5ef4d7cb795e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:06:58.712126 3713 consensus_queue.cc:1035] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 513daa018ebf4514894d5ef4d7cb795e [LEADER]: Connected to new peer: Peer: permanent_uuid: "130105c8e4b94acea85e45c8cbbfd8ad" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:06:58.712548 3710 consensus_queue.cc:1035] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 513daa018ebf4514894d5ef4d7cb795e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae6436ae2ee446eacad8af409ce0daf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:06:58.713852 3459 heartbeater.cc:503] Master 127.27.225.253:34129 was elected leader, sending a full tablet report...
I20260806 21:06:58.715062 3709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.714728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.195:32873)
0806 21:06:58.714874 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.714948 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 21:06:58.715144 3707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.714740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.194:57619)
0806 21:06:58.714984 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.715041 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175}
I20260806 21:06:58.719378 3712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.718984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.193:58657)
0806 21:06:58.719225 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.719292 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260806 21:06:58.720752 3707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.720413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.194:45269)
0806 21:06:58.720610 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.720675 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260806 21:06:58.722083 3709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.721856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:45179 (local address 127.27.225.195:44981)
0806 21:06:58.721953 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.722012 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 21:06:58.731254 2946 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:06:58.731485 2946 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:06:58.736338 2946 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4859fba6ee99437aa8a0285c7270fe12: Sending DeleteTablet for 3 replicas of tablet dc9962ef9b894b8fa8bfd4f9a268abd9
I20260806 21:06:58.737231 3528 tablet_service.cc:1514] Processing DeleteTablet for tablet dc9962ef9b894b8fa8bfd4f9a268abd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:58 UTC) from {username='oryx', principal='oryx/127.27.225.253@KRBTEST.COM'} at 127.0.0.1:51488
I20260806 21:06:58.737344 3250 tablet_service.cc:1514] Processing DeleteTablet for tablet dc9962ef9b894b8fa8bfd4f9a268abd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:58 UTC) from {username='oryx', principal='oryx/127.27.225.253@KRBTEST.COM'} at 127.0.0.1:54302
I20260806 21:06:58.737857 3390 tablet_service.cc:1514] Processing DeleteTablet for tablet dc9962ef9b894b8fa8bfd4f9a268abd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:06:58 UTC) from {username='oryx', principal='oryx/127.27.225.253@KRBTEST.COM'} at 127.0.0.1:34028
I20260806 21:06:58.738396 3727 tablet_replica.cc:330] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 3ae6436ae2ee446eacad8af409ce0daf: stopping tablet replica
I20260806 21:06:58.738544 3727 raft_consensus.cc:2229] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 3ae6436ae2ee446eacad8af409ce0daf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:06:58.738582 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 3095
I20260806 21:06:58.738715 3727 raft_consensus.cc:2258] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 3ae6436ae2ee446eacad8af409ce0daf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:58.739715 3729 tablet_replica.cc:330] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 130105c8e4b94acea85e45c8cbbfd8ad: stopping tablet replica
I20260806 21:06:58.739866 3729 raft_consensus.cc:2229] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 130105c8e4b94acea85e45c8cbbfd8ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:06:58.740018 3729 raft_consensus.cc:2258] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 130105c8e4b94acea85e45c8cbbfd8ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:06:58.740922 3729 ts_tablet_manager.cc:1905] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 130105c8e4b94acea85e45c8cbbfd8ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:58.742692 3727 ts_tablet_manager.cc:1905] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 3ae6436ae2ee446eacad8af409ce0daf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:06:58.743815 3729 ts_tablet_manager.cc:1918] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 130105c8e4b94acea85e45c8cbbfd8ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:06:58.743942 3729 log.cc:1196] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 130105c8e4b94acea85e45c8cbbfd8ad: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-1/wal/wals/dc9962ef9b894b8fa8bfd4f9a268abd9
I20260806 21:06:58.744242 3729 ts_tablet_manager.cc:1939] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 130105c8e4b94acea85e45c8cbbfd8ad: Deleting consensus metadata
I20260806 21:06:58.745126 2931 catalog_manager.cc:4841] TS 130105c8e4b94acea85e45c8cbbfd8ad (127.27.225.194:42173): tablet dc9962ef9b894b8fa8bfd4f9a268abd9 (table test-table [id=ec13d3a927594c79b6bcaa8c18debfa9]) successfully deleted
I20260806 21:06:58.746259 3727 ts_tablet_manager.cc:1918] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 3ae6436ae2ee446eacad8af409ce0daf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:06:58.746364 3727 log.cc:1196] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 3ae6436ae2ee446eacad8af409ce0daf: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/dc9962ef9b894b8fa8bfd4f9a268abd9
I20260806 21:06:58.746687 3727 ts_tablet_manager.cc:1939] T dc9962ef9b894b8fa8bfd4f9a268abd9 P 3ae6436ae2ee446eacad8af409ce0daf: Deleting consensus metadata
I20260806 21:06:58.747677 2934 catalog_manager.cc:4841] TS 3ae6436ae2ee446eacad8af409ce0daf (127.27.225.195:43009): tablet dc9962ef9b894b8fa8bfd4f9a268abd9 (table test-table [id=ec13d3a927594c79b6bcaa8c18debfa9]) successfully deleted
I20260806 21:06:58.749044 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 3326
I20260806 21:06:58.749195 3695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:06:58.748882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:43373 (local address 127.0.0.1:54314)
0806 21:06:58.749030 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 21:06:58.749089 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.193:43373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 21:06:58.750288 2932 catalog_manager.cc:4584] TS 513daa018ebf4514894d5ef4d7cb795e (127.27.225.193:43373): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dc9962ef9b894b8fa8bfd4f9a268abd9: Network error: Client connection negotiation failed: client connection to 127.27.225.193:43373: connect: Connection refused (error 111)
I20260806 21:06:58.756059 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 3464
I20260806 21:06:58.764596 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 2810
I20260806 21:06:58.771490 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 2913
2026-08-06T21:06:58Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7597 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:06:58 dist-test-slave-69rg krb5kdc[3735](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:06:58 dist-test-slave-69rg krb5kdc[3735](info): set up 2 sockets
Aug 06 21:06:58 dist-test-slave-69rg krb5kdc[3735](info): commencing operation
krb5kdc: starting...
W20260806 21:06:59.834237 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:06:59 dist-test-slave-69rg krb5kdc[3735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:06:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:06:59Z Disabled control of system clock
WARNING: no policy specified for oryx/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.254.keytab.
Entry for principal oryx/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.254.keytab.
I20260806 21:06:59.972901 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:33337
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.254.keytab
--principal=oryx/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:41333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:33337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:07:00.106627 3751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:07:00.106959 3751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:07:00.107039 3751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:07:00.111320 3751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:07:00.111415 3751 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 21:07:00.111449 3751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:07:00.111490 3751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:07:00.111523 3751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:07:00.111557 3751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:07:00.117307 3751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:41333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:33337
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.254.keytab
--principal=oryx/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:33337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:07:00.118717 3751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:07:00.119792 3751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:07:00.128813 3759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:00.128868 3751 server_base.cc:1034] running on GCE node
W20260806 21:07:00.128841 3756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:00.128813 3757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:00.129864 3751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:07:00.131109 3751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:07:00.132292 3751 hybrid_clock.cc:648] HybridClock initialized: now 1786050420132274 us; error 36 us; skew 500 ppm
Aug 06 21:07:00 dist-test-slave-69rg krb5kdc[3735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050420, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:07:00.140930 3751 init.cc:377] Logged in from keytab as oryx/127.27.225.254@KRBTEST.COM (short username oryx)
I20260806 21:07:00.142113 3751 webserver.cc:460] Webserver started at http://127.27.225.254:42901/ using document root <none> and password file <none>
I20260806 21:07:00.142685 3751 fs_manager.cc:359] Metadata directory not provided
I20260806 21:07:00.142747 3751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:07:00.142937 3751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:07:00.145084 3751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "6ef1adaf1849477cb101f0574cc34f5f"
format_stamp: "Formatted at 2026-08-06 21:07:00 on dist-test-slave-69rg"
server_key: "58e5da071de2a66c26580095c175d144"
server_key_iv: "36907be9d809a576f2a5dec61b5d5d08"
server_key_version: "encryptionkey@0"
I20260806 21:07:00.145622 3751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "6ef1adaf1849477cb101f0574cc34f5f"
format_stamp: "Formatted at 2026-08-06 21:07:00 on dist-test-slave-69rg"
server_key: "58e5da071de2a66c26580095c175d144"
server_key_iv: "36907be9d809a576f2a5dec61b5d5d08"
server_key_version: "encryptionkey@0"
I20260806 21:07:00.149596 3751 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 21:07:00.152340 3766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:00.153970 3751 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:07:00.154098 3751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "6ef1adaf1849477cb101f0574cc34f5f"
format_stamp: "Formatted at 2026-08-06 21:07:00 on dist-test-slave-69rg"
server_key: "58e5da071de2a66c26580095c175d144"
server_key_iv: "36907be9d809a576f2a5dec61b5d5d08"
server_key_version: "encryptionkey@0"
I20260806 21:07:00.154240 3751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:07:00.174352 3751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:07:00.177937 3751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:07:00.178124 3751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:07:00.186764 3751 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:33337
I20260806 21:07:00.186802 3830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:33337 every 8 connection(s)
I20260806 21:07:00.187758 3751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:07:00.190686 3832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:07:00.191294 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 3751
I20260806 21:07:00.191431 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:07:00.191752 28551 external_mini_cluster.cc:1428] Setting key 72cff02d37c88c460c722abfeb5ffb6e
I20260806 21:07:00.197021 3832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f: Bootstrap starting.
I20260806 21:07:00.199592 3832 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f: Neither blocks nor log segments found. Creating new log.
I20260806 21:07:00.200384 3832 log.cc:824] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f: Log is configured to *not* fsync() on all Append() calls
Aug 06 21:07:00 dist-test-slave-69rg krb5kdc[3735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM
I20260806 21:07:00.202531 3832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f: No bootstrap required, opened a new log
I20260806 21:07:00.206314 3832 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ef1adaf1849477cb101f0574cc34f5f" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 33337 } }
I20260806 21:07:00.206614 3832 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:07:00.206712 3832 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ef1adaf1849477cb101f0574cc34f5f, State: Initialized, Role: FOLLOWER
I20260806 21:07:00.207273 3832 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ef1adaf1849477cb101f0574cc34f5f" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 33337 } }
I20260806 21:07:00.207449 3832 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:07:00.207531 3832 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:07:00.207650 3832 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:07:00.208429 3835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:00.193260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51832 (local address 127.27.225.254:33337)
0806 21:07:00.193733 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:00.193748 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:00.193802 (+ 54us) server_negotiation.cc:407] Connection header received
0806 21:07:00.194529 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:00.194546 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:00.194884 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:00.195235 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:00.196343 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:00.197590 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:00.198879 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:00.199382 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:00.199573 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:00.202407 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:00.202412 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:00.202432 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:07:00.202461 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:00.205231 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:00.205762 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:00.205768 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:00.205773 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:00.205960 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:00.206258 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:00.206262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:00.206264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:00.206745 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:00.206961 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:00.207260 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 21:07:00.207570 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":147,"threads_started":1}
I20260806 21:07:00.208731 3832 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ef1adaf1849477cb101f0574cc34f5f" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 33337 } }
I20260806 21:07:00.209116 3832 leader_election.cc:302] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ef1adaf1849477cb101f0574cc34f5f; no voters:
I20260806 21:07:00.209440 3832 leader_election.cc:288] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:07:00.209568 3837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:07:00.209776 3837 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f [term 1 LEADER]: Becoming Leader. State: Replica: 6ef1adaf1849477cb101f0574cc34f5f, State: Running, Role: LEADER
I20260806 21:07:00.210096 3837 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ef1adaf1849477cb101f0574cc34f5f" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 33337 } }
I20260806 21:07:00.210804 3832 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:07:00.211665 3838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ef1adaf1849477cb101f0574cc34f5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ef1adaf1849477cb101f0574cc34f5f" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 33337 } } }
I20260806 21:07:00.211814 3838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f [sys.catalog]: This master's current role is: LEADER
I20260806 21:07:00.212169 3839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ef1adaf1849477cb101f0574cc34f5f. Latest consensus state: current_term: 1 leader_uuid: "6ef1adaf1849477cb101f0574cc34f5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ef1adaf1849477cb101f0574cc34f5f" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 33337 } } }
I20260806 21:07:00.212239 3846 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:07:00.212265 3839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f [sys.catalog]: This master's current role is: LEADER
I20260806 21:07:00.215512 3846 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:07:00.221637 3846 catalog_manager.cc:1347] Generated new cluster ID: 995f1031ad37459fb9856a7f0f1eeeb9
I20260806 21:07:00.221724 3846 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:07:00.233886 3846 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:07:00.235222 3846 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:07:00.240831 3846 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f: Generated new TSK 0
I20260806 21:07:00.241652 3846 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.27.225.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.225.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.225.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.193.keytab.
Entry for principal oryx/127.27.225.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.193.keytab.
I20260806 21:07:00.274178 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:33337
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.193.keytab
--principal=oryx/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:41333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:07:00.396464 3858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:07:00.396752 3858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:07:00.396798 3858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:07:00.401062 3858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:07:00.401121 3858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:07:00.401166 3858 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 21:07:00.401247 3858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:07:00.406647 3858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:41333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.193.keytab
--principal=oryx/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.225.254:33337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:07:00.407810 3858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:07:00.408879 3858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:07:00.419323 3863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:00.419302 3864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:00.419577 3858 server_base.cc:1034] running on GCE node
W20260806 21:07:00.419302 3866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:00.420342 3858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:07:00.421099 3858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:07:00.422365 3858 hybrid_clock.cc:648] HybridClock initialized: now 1786050420422341 us; error 39 us; skew 500 ppm
Aug 06 21:07:00 dist-test-slave-69rg krb5kdc[3735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050420, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:07:00.425272 3858 init.cc:377] Logged in from keytab as oryx/127.27.225.193@KRBTEST.COM (short username oryx)
I20260806 21:07:00.426262 3858 webserver.cc:460] Webserver started at http://127.27.225.193:41933/ using document root <none> and password file <none>
I20260806 21:07:00.426826 3858 fs_manager.cc:359] Metadata directory not provided
I20260806 21:07:00.426918 3858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:07:00.427165 3858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:07:00.429252 3858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "871ebb27b0794ec0a464ba6e581a08b8"
format_stamp: "Formatted at 2026-08-06 21:07:00 on dist-test-slave-69rg"
server_key: "3ee0a66349b81d586d5c8fdd3c5d2e86"
server_key_iv: "bbfac54c2bbd030d2b5f066e1b6e7c46"
server_key_version: "encryptionkey@0"
I20260806 21:07:00.429853 3858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "871ebb27b0794ec0a464ba6e581a08b8"
format_stamp: "Formatted at 2026-08-06 21:07:00 on dist-test-slave-69rg"
server_key: "3ee0a66349b81d586d5c8fdd3c5d2e86"
server_key_iv: "bbfac54c2bbd030d2b5f066e1b6e7c46"
server_key_version: "encryptionkey@0"
I20260806 21:07:00.433511 3858 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:07:00.435959 3873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:00.437182 3858 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:07:00.437318 3858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "871ebb27b0794ec0a464ba6e581a08b8"
format_stamp: "Formatted at 2026-08-06 21:07:00 on dist-test-slave-69rg"
server_key: "3ee0a66349b81d586d5c8fdd3c5d2e86"
server_key_iv: "bbfac54c2bbd030d2b5f066e1b6e7c46"
server_key_version: "encryptionkey@0"
I20260806 21:07:00.437431 3858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:07:00.452539 3858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:07:00.456048 3858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:07:00.456291 3858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:07:00.457933 3858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:07:00.458009 3858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:00.458119 3858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:07:00.458180 3858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 21:07:00 dist-test-slave-69rg krb5kdc[3735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050420, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM
I20260806 21:07:00.469280 3858 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:32999
I20260806 21:07:00.469496 3990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:32999 every 8 connection(s)
I20260806 21:07:00.470450 3858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:07:00.472295 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 3858
I20260806 21:07:00.472437 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:07:00.472783 28551 external_mini_cluster.cc:1428] Setting key 14ca8c49639237724776a5f7167704ac
I20260806 21:07:00.473897 3835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:00.458749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:48209 (local address 127.27.225.254:33337)
0806 21:07:00.458880 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:00.458885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:00.459877 (+ 992us) server_negotiation.cc:407] Connection header received
0806 21:07:00.462020 (+ 2143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:00.462024 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:00.462089 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:00.462195 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:00.463671 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:00.464321 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:00.466220 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:00.466466 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:00.466551 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:00.469949 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:00.469954 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:00.469957 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:07:00.469990 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:00.471615 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:00.472146 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:00.472149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:00.472150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:00.472227 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:00.472620 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:00.472623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:00.472624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:00.472838 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:00.472939 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:00.473557 (+ 618us) server_negotiation.cc:299] Negotiation successful
0806 21:07:00.473696 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 21:07:00.474777 3882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:00.459227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:33337 (local address 127.27.225.193:48209)
0806 21:07:00.459684 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:00.459746 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:00.460796 (+ 1050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:00.462467 (+ 1671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:00.462477 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:00.462956 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:00.463517 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:00.463534 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:00.464475 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:00.464479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:00.466098 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:00.466110 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:00.466654 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:00.466659 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:00.466899 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:00.467711 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:00.467740 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:00.469742 (+ 2002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:07:00.471739 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:00.471743 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:00.471754 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:00.472049 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:00.472323 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:00.472328 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:00.472330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:00.472523 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:00.472940 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:07:00.472948 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:07:00.473266 (+ 318us) client_negotiation.cc:769] Sending connection context
0806 21:07:00.473493 (+ 227us) client_negotiation.cc:240] Negotiation successful
0806 21:07:00.473813 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":135,"threads_started":1}
I20260806 21:07:00.476116 3991 heartbeater.cc:347] Connected to a master server at 127.27.225.254:33337
I20260806 21:07:00.476282 3991 heartbeater.cc:464] Registering TS with master...
I20260806 21:07:00.476852 3991 heartbeater.cc:511] Master 127.27.225.254:33337 requested a full tablet report, sending...
I20260806 21:07:00.478466 3784 ts_manager.cc:194] Registered new tserver with Master: 871ebb27b0794ec0a464ba6e581a08b8 (127.27.225.193:32999)
I20260806 21:07:00.479590 3784 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.225.193@KRBTEST.COM'} at 127.27.225.193:48209
WARNING: no policy specified for oryx/127.27.225.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.225.194@KRBTEST.COM" created.
I20260806 21:07:00.489128 3835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:00.478488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:47905 (local address 127.27.225.254:33337)
0806 21:07:00.478606 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:00.478610 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:00.482123 (+ 3513us) server_negotiation.cc:407] Connection header received
0806 21:07:00.482278 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:00.482281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:00.482333 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:00.482414 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:00.482884 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:00.483417 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:00.484323 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:00.484568 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:00.484677 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:00.485890 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:00.485894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:00.485896 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:07:00.485921 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:00.487407 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:00.487859 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:00.487862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:00.487864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:00.487949 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:00.488242 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:00.488246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:00.488248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:00.488441 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:00.488551 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:00.488894 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 21:07:00.489006 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 21:07:00.489252 3998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:00.478396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:33337 (local address 127.27.225.193:47905)
0806 21:07:00.482024 (+ 3628us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:00.482038 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:00.482163 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:00.482434 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:00.482437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:00.482657 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:00.482762 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:00.482768 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:00.483550 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:00.483553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:00.484190 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:00.484204 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:00.484696 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:00.484700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:00.484804 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:00.485372 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:00.485384 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:00.485725 (+ 341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:07:00.487527 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:00.487531 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:00.487533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:00.487754 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:00.488046 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:00.488049 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:00.488051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:00.488143 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:00.488539 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:07:00.488542 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:07:00.488648 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 21:07:00.488786 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 21:07:00.488967 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3545,"thread_start_us":113,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.225.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.194.keytab.
Entry for principal oryx/127.27.225.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.194.keytab.
I20260806 21:07:00.504765 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:33337
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.194.keytab
--principal=oryx/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:41333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:07:00.619656 4000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:07:00.619966 4000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:07:00.620040 4000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:07:00.624140 4000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:07:00.624230 4000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:07:00.624301 4000 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 21:07:00.624387 4000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:07:00.629760 4000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:41333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.194.keytab
--principal=oryx/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.225.254:33337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:07:00.630925 4000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:07:00.631955 4000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:07:00.641848 4009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:00.641852 4006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:00.641853 4007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:00.642513 4000 server_base.cc:1034] running on GCE node
I20260806 21:07:00.642957 4000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:07:00.643659 4000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:07:00.644824 4000 hybrid_clock.cc:648] HybridClock initialized: now 1786050420644790 us; error 53 us; skew 500 ppm
Aug 06 21:07:00 dist-test-slave-69rg krb5kdc[3735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050420, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:07:00.647763 4000 init.cc:377] Logged in from keytab as oryx/127.27.225.194@KRBTEST.COM (short username oryx)
I20260806 21:07:00.648665 4000 webserver.cc:460] Webserver started at http://127.27.225.194:44989/ using document root <none> and password file <none>
I20260806 21:07:00.649195 4000 fs_manager.cc:359] Metadata directory not provided
I20260806 21:07:00.649255 4000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:07:00.649437 4000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:07:00.651626 4000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "ff6219ada81d4c209dc32acc02b15831"
format_stamp: "Formatted at 2026-08-06 21:07:00 on dist-test-slave-69rg"
server_key: "f97578b727618e8c238e76e901e4a153"
server_key_iv: "e73bddbb64cfaa052be48692ef3fbe2a"
server_key_version: "encryptionkey@0"
I20260806 21:07:00.652226 4000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "ff6219ada81d4c209dc32acc02b15831"
format_stamp: "Formatted at 2026-08-06 21:07:00 on dist-test-slave-69rg"
server_key: "f97578b727618e8c238e76e901e4a153"
server_key_iv: "e73bddbb64cfaa052be48692ef3fbe2a"
server_key_version: "encryptionkey@0"
I20260806 21:07:00.656402 4000 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 21:07:00.659251 4016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:00.660564 4000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 21:07:00.660705 4000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "ff6219ada81d4c209dc32acc02b15831"
format_stamp: "Formatted at 2026-08-06 21:07:00 on dist-test-slave-69rg"
server_key: "f97578b727618e8c238e76e901e4a153"
server_key_iv: "e73bddbb64cfaa052be48692ef3fbe2a"
server_key_version: "encryptionkey@0"
I20260806 21:07:00.660825 4000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:07:00.687923 4000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:07:00.691596 4000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:07:00.691818 4000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:07:00.693291 4000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:07:00.693372 4000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:00.693445 4000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:07:00.693498 4000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 21:07:00 dist-test-slave-69rg krb5kdc[3735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050420, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM
I20260806 21:07:00.704974 4000 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:35821
I20260806 21:07:00.705080 4132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:35821 every 8 connection(s)
I20260806 21:07:00.706215 4000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:07:00.708067 3835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:00.694467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:51087 (local address 127.27.225.254:33337)
0806 21:07:00.694604 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:00.694607 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:00.695679 (+ 1072us) server_negotiation.cc:407] Connection header received
0806 21:07:00.696914 (+ 1235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:00.696918 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:00.696982 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:00.697095 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:00.698850 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:00.699504 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:00.700688 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:00.701059 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:00.701188 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:00.703889 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:00.703892 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:00.703896 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:07:00.703929 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:00.705617 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:00.706234 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:00.706239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:00.706241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:00.706348 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:00.706716 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:00.706720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:00.706722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:00.706974 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:00.707077 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:00.707672 (+ 595us) server_negotiation.cc:299] Negotiation successful
0806 21:07:00.707843 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 21:07:00.708737 4025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:00.695038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:33337 (local address 127.27.225.194:51087)
0806 21:07:00.695528 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:00.695571 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:00.696662 (+ 1091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:00.697307 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:00.697320 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:00.697860 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:00.698644 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:00.698668 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:00.699645 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:00.699649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:00.700541 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:00.700550 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:00.701251 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:00.701256 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:00.701479 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:00.702156 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:00.702172 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:00.703696 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:07:00.705772 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:00.705776 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:00.705788 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:00.706114 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:00.706457 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:00.706460 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:00.706461 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:00.706598 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:00.707138 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:07:00.707143 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:07:00.707376 (+ 233us) client_negotiation.cc:769] Sending connection context
0806 21:07:00.707528 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 21:07:00.707803 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":133,"threads_started":1}
I20260806 21:07:00.710191 4133 heartbeater.cc:347] Connected to a master server at 127.27.225.254:33337
I20260806 21:07:00.710357 4133 heartbeater.cc:464] Registering TS with master...
I20260806 21:07:00.710983 4133 heartbeater.cc:511] Master 127.27.225.254:33337 requested a full tablet report, sending...
I20260806 21:07:00.712311 3783 ts_manager.cc:194] Registered new tserver with Master: ff6219ada81d4c209dc32acc02b15831 (127.27.225.194:35821)
I20260806 21:07:00.713068 3783 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.225.194@KRBTEST.COM'} at 127.27.225.194:51087
I20260806 21:07:00.713305 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 4000
I20260806 21:07:00.713436 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:07:00.713778 28551 external_mini_cluster.cc:1428] Setting key d35f529d0d4ba4a609a45cc32bce8b79
I20260806 21:07:00.723697 4139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:00.712757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:33337 (local address 127.27.225.194:46949)
0806 21:07:00.713530 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:00.713545 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:00.713650 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:00.715547 (+ 1897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:00.715549 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:00.715787 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:00.715886 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:00.715892 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:00.716804 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:00.716807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:00.717590 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:00.717606 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:00.718243 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:00.718248 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:00.718390 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:00.719115 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:00.719127 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:00.719486 (+ 359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:07:00.721730 (+ 2244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:00.721733 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:00.721735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:00.722087 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:00.722474 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:00.722477 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:00.722478 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:00.722574 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:00.723203 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:07:00.723205 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:07:00.723294 (+ 89us) client_negotiation.cc:769] Sending connection context
0806 21:07:00.723381 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 21:07:00.723518 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":109,"threads_started":1}
I20260806 21:07:00.723826 3835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:00.712875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:46949 (local address 127.27.225.254:33337)
0806 21:07:00.713005 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:00.713010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:00.713636 (+ 626us) server_negotiation.cc:407] Connection header received
0806 21:07:00.715306 (+ 1670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:00.715309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:00.715370 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:00.715478 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:00.716029 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:00.716657 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:00.717744 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:00.718058 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:00.718207 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:00.719643 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:00.719649 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:00.719656 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:07:00.719690 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:00.721578 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:00.722210 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:00.722216 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:00.722220 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:00.722331 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:00.722722 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:00.722727 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:00.722732 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:00.723094 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:00.723212 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:00.723474 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 21:07:00.723635 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
WARNING: no policy specified for oryx/127.27.225.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.225.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.225.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.195.keytab.
Entry for principal oryx/127.27.225.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.195.keytab.
I20260806 21:07:00.746573 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:33337
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.195.keytab
--principal=oryx/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:41333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:07:00.880411 4142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:07:00.882086 4142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:07:00.882550 4142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:07:00.891297 4142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:07:00.891449 4142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:07:00.891506 4142 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 21:07:00.891557 4142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:07:00.898938 4142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:41333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.195.keytab
--principal=oryx/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.225.254:33337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:07:00.900126 4142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:07:00.901427 4142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:07:00.915506 4148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:00.915871 4150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:00.915813 4142 server_base.cc:1034] running on GCE node
W20260806 21:07:00.915829 4147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:00.916414 4142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:07:00.917112 4142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:07:00.920718 4142 hybrid_clock.cc:648] HybridClock initialized: now 1786050420920714 us; error 150 us; skew 500 ppm
Aug 06 21:07:00 dist-test-slave-69rg krb5kdc[3735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050420, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:07:00.925405 4142 init.cc:377] Logged in from keytab as oryx/127.27.225.195@KRBTEST.COM (short username oryx)
I20260806 21:07:00.927215 4142 webserver.cc:460] Webserver started at http://127.27.225.195:35575/ using document root <none> and password file <none>
I20260806 21:07:00.928470 4142 fs_manager.cc:359] Metadata directory not provided
I20260806 21:07:00.928695 4142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:07:00.928984 4142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:07:00.931403 4142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "0b706acb694f4024acecc217a6c7a3b5"
format_stamp: "Formatted at 2026-08-06 21:07:00 on dist-test-slave-69rg"
server_key: "9cf18cebab8fb4fa6da3eb8f429a9680"
server_key_iv: "c3c5df09d3a277a73c1e0d03167396ef"
server_key_version: "encryptionkey@0"
I20260806 21:07:00.932068 4142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "0b706acb694f4024acecc217a6c7a3b5"
format_stamp: "Formatted at 2026-08-06 21:07:00 on dist-test-slave-69rg"
server_key: "9cf18cebab8fb4fa6da3eb8f429a9680"
server_key_iv: "c3c5df09d3a277a73c1e0d03167396ef"
server_key_version: "encryptionkey@0"
I20260806 21:07:00.937736 4142 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 21:07:00.942334 4157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:00.945039 4142 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.003s
I20260806 21:07:00.945266 4142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "0b706acb694f4024acecc217a6c7a3b5"
format_stamp: "Formatted at 2026-08-06 21:07:00 on dist-test-slave-69rg"
server_key: "9cf18cebab8fb4fa6da3eb8f429a9680"
server_key_iv: "c3c5df09d3a277a73c1e0d03167396ef"
server_key_version: "encryptionkey@0"
I20260806 21:07:00.945668 4142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:07:00.974136 4142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:07:00.978312 4142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:07:00.978540 4142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:07:00.980078 4142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:07:00.980160 4142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:00.980235 4142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:07:00.980266 4142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:00.990689 4142 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:44843
I20260806 21:07:00.990769 4280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:44843 every 8 connection(s)
Aug 06 21:07:00 dist-test-slave-69rg krb5kdc[3735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050420, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM
I20260806 21:07:00.991734 4142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:07:00.996747 3835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:00.981185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:38275 (local address 127.27.225.254:33337)
0806 21:07:00.981347 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:00.981351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:00.982477 (+ 1126us) server_negotiation.cc:407] Connection header received
0806 21:07:00.983696 (+ 1219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:00.983699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:00.983763 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:00.983882 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:00.985546 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:00.986621 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:00.988245 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:00.988676 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:00.988818 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:00.991871 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:00.991875 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:00.991878 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:07:00.991952 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:00.994194 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:00.994820 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:00.994823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:00.994825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:00.994925 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:00.995293 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:00.995297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:00.995298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:00.995550 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:00.995674 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:00.996442 (+ 768us) server_negotiation.cc:299] Negotiation successful
0806 21:07:00.996582 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 21:07:00.997323 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 4142
I20260806 21:07:00.997493 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:07:00.997710 4166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:00.981687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:33337 (local address 127.27.225.195:38275)
0806 21:07:00.982306 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:00.982350 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:00.983451 (+ 1101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:00.984058 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:00.984067 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:00.984589 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:00.985341 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:00.985364 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:00.986775 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:00.986779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:00.988093 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:00.988104 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:00.988883 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:00.988888 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:00.989132 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:00.989950 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:00.989977 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:00.991656 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:07:00.994358 (+ 2702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:00.994364 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:00.994377 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:00.994686 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:00.995036 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:00.995038 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:00.995040 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:00.995190 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:00.995724 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:07:00.995731 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:07:00.996124 (+ 393us) client_negotiation.cc:769] Sending connection context
0806 21:07:00.996331 (+ 207us) client_negotiation.cc:240] Negotiation successful
0806 21:07:00.996631 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":162,"threads_started":1}
I20260806 21:07:00.997915 28551 external_mini_cluster.cc:1428] Setting key b6dba6c181a59ed04789c1a568b0bcaa
I20260806 21:07:01.000085 4281 heartbeater.cc:347] Connected to a master server at 127.27.225.254:33337
I20260806 21:07:01.000275 4281 heartbeater.cc:464] Registering TS with master...
I20260806 21:07:01.001096 4281 heartbeater.cc:511] Master 127.27.225.254:33337 requested a full tablet report, sending...
I20260806 21:07:01.002856 3782 ts_manager.cc:194] Registered new tserver with Master: 0b706acb694f4024acecc217a6c7a3b5 (127.27.225.195:44843)
I20260806 21:07:01.004534 3782 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.225.195@KRBTEST.COM'} at 127.27.225.195:38275
I20260806 21:07:01.006156 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:07:01.016526 3835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.003703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:60823 (local address 127.27.225.254:33337)
0806 21:07:01.003945 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:01.003951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:01.004306 (+ 355us) server_negotiation.cc:407] Connection header received
0806 21:07:01.004500 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:01.004503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:01.004550 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:01.004656 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:01.005899 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.006983 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.008108 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.008344 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.008466 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.010881 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:01.010890 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:01.010897 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:07:01.010939 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:01.014002 (+ 3063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:01.014617 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:01.014622 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:01.014625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:01.014775 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:01.015331 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:01.015335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:01.015336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:01.015571 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:01.015695 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:01.016230 (+ 535us) server_negotiation.cc:299] Negotiation successful
0806 21:07:01.016388 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260806 21:07:01.016913 4287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.003483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:33337 (local address 127.27.225.195:60823)
0806 21:07:01.004158 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:01.004176 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:01.004359 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:01.004686 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:01.004689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:01.005275 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:01.005708 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:01.005718 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.007166 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.007169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:01.007990 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:01.008000 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.008486 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.008490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:01.008586 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.009531 (+ 945us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:01.009710 (+ 179us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:01.010607 (+ 897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:07:01.014201 (+ 3594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:01.014205 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:01.014206 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:01.014478 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:01.015026 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:01.015030 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:01.015033 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:01.015153 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:01.015698 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:07:01.015704 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:07:01.015846 (+ 142us) client_negotiation.cc:769] Sending connection context
0806 21:07:01.016005 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 21:07:01.016215 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":134,"threads_started":1}
Aug 06 21:07:01 dist-test-slave-69rg krb5kdc[3735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050421, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 21:07:01 dist-test-slave-69rg krb5kdc[3735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:01 dist-test-slave-69rg krb5kdc[3735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 21:07:01.041064 3835 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:01.033107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51834 (local address 127.27.225.254:33337)
0806 21:07:01.033317 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:01.033321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:01.033336 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:07:01.033380 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:01.033382 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:01.033423 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:01.033504 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:01.034862 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.035430 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.036512 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.036761 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.036856 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.040931 (+ 4075us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51834: BlockingRecv error: recv got EOF from 127.0.0.1:51834 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 06 21:07:01 dist-test-slave-69rg krb5kdc[3735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050421, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM
I20260806 21:07:01.052186 3835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.042998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51836 (local address 127.27.225.254:33337)
0806 21:07:01.043165 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:01.043169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:01.043184 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:07:01.043333 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:01.043336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:01.043383 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:01.043464 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:01.044159 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.044708 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.045530 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.045779 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.045957 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.048763 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:01.048767 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:01.048769 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:07:01.048797 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:01.050465 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:01.050929 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:01.050932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:01.050933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:01.051018 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:01.051350 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:01.051352 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:01.051353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:01.051549 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:01.051670 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:01.051942 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 21:07:01.052053 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 21:07:01.055673 3784 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51836:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:07:01.059235 3784 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:07:01.076227 4305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.069593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51548 (local address 127.27.225.193:32999)
0806 21:07:01.070071 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:01.070230 (+ 159us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:01.070553 (+ 323us) server_negotiation.cc:407] Connection header received
0806 21:07:01.070883 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:01.070888 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:01.071092 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:01.071190 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:07:01.071637 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.072886 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.074656 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.075259 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.075411 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.075572 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:01.075927 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 21:07:01.076056 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":93,"threads_started":1}
I20260806 21:07:01.076288 4303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.070160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:35821 (local address 127.0.0.1:57138)
0806 21:07:01.070825 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:01.070838 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:01.070905 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:01.071688 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:01.071694 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:01.071719 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:07:01.071782 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:01.071786 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.072866 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.072872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:01.074326 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:01.074336 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.075795 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.075799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:01.075927 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.075955 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 21:07:01.076014 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 21:07:01.076147 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"mutex_wait_us":254,"thread_start_us":96,"threads_started":1}
I20260806 21:07:01.076669 4307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.069384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:32999 (local address 127.0.0.1:51548)
0806 21:07:01.070444 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:01.070462 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:01.070668 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:01.071356 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:01.071361 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:01.071379 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:07:01.071514 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:01.071519 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.073007 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.073011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:01.074475 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:01.074487 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.075401 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.075405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:01.075529 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.075570 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 21:07:01.076416 (+ 846us) client_negotiation.cc:240] Negotiation successful
0806 21:07:01.076495 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":986,"mutex_wait_us":14,"spinlock_wait_cycles":1152,"thread_start_us":123,"threads_started":1}
I20260806 21:07:01.076963 4306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.069605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45390 (local address 127.27.225.195:44843)
0806 21:07:01.070069 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:01.070085 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:01.070547 (+ 462us) server_negotiation.cc:407] Connection header received
0806 21:07:01.070866 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:01.070874 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:01.071092 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:01.071228 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:07:01.071731 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.072890 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.074613 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.075601 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.075796 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.075992 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:01.076669 (+ 677us) server_negotiation.cc:299] Negotiation successful
0806 21:07:01.076796 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":119,"threads_started":1}
I20260806 21:07:01.076997 4308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.070334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57138 (local address 127.27.225.194:35821)
0806 21:07:01.071213 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:01.071229 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:01.071250 (+ 21us) server_negotiation.cc:407] Connection header received
0806 21:07:01.071322 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:01.071327 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:01.071503 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:01.071896 (+ 393us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:07:01.072003 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.072736 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.074460 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.075638 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.075886 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.076017 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:01.076075 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 21:07:01.076173 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":756,"thread_start_us":123,"threads_started":1}
I20260806 21:07:01.077373 4304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.069324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:44843 (local address 127.0.0.1:45390)
0806 21:07:01.070400 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:01.070459 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:01.070667 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:01.071407 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:01.071410 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:01.071434 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:07:01.071543 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:01.071549 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.073151 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.073157 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:01.074475 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:01.074487 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.075791 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.075797 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:01.076494 (+ 697us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.076515 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 21:07:01.077164 (+ 649us) client_negotiation.cc:240] Negotiation successful
0806 21:07:01.077246 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":890,"mutex_wait_us":108,"spinlock_wait_cycles":3584,"thread_start_us":139,"threads_started":1}
I20260806 21:07:01.080235 3924 tablet_service.cc:1467] Processing CreateTablet for tablet 0913d0f9b1d342b580425f5a4e4b3342 (DEFAULT_TABLE table=test-table [id=12a9c35a87a04206a9b6e6c576b52432]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:07:01.080235 4066 tablet_service.cc:1467] Processing CreateTablet for tablet 0913d0f9b1d342b580425f5a4e4b3342 (DEFAULT_TABLE table=test-table [id=12a9c35a87a04206a9b6e6c576b52432]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:07:01.080230 4206 tablet_service.cc:1467] Processing CreateTablet for tablet 0913d0f9b1d342b580425f5a4e4b3342 (DEFAULT_TABLE table=test-table [id=12a9c35a87a04206a9b6e6c576b52432]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:07:01.081478 4066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0913d0f9b1d342b580425f5a4e4b3342. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:07:01.081478 3924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0913d0f9b1d342b580425f5a4e4b3342. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:07:01.081563 4206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0913d0f9b1d342b580425f5a4e4b3342. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:07:01.089321 4309 tablet_bootstrap.cc:492] T 0913d0f9b1d342b580425f5a4e4b3342 P 0b706acb694f4024acecc217a6c7a3b5: Bootstrap starting.
I20260806 21:07:01.091173 4310 tablet_bootstrap.cc:492] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8: Bootstrap starting.
I20260806 21:07:01.092141 4309 tablet_bootstrap.cc:654] T 0913d0f9b1d342b580425f5a4e4b3342 P 0b706acb694f4024acecc217a6c7a3b5: Neither blocks nor log segments found. Creating new log.
I20260806 21:07:01.093096 4309 log.cc:824] T 0913d0f9b1d342b580425f5a4e4b3342 P 0b706acb694f4024acecc217a6c7a3b5: Log is configured to *not* fsync() on all Append() calls
I20260806 21:07:01.093432 4310 tablet_bootstrap.cc:654] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8: Neither blocks nor log segments found. Creating new log.
I20260806 21:07:01.094017 4311 tablet_bootstrap.cc:492] T 0913d0f9b1d342b580425f5a4e4b3342 P ff6219ada81d4c209dc32acc02b15831: Bootstrap starting.
I20260806 21:07:01.094295 4310 log.cc:824] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8: Log is configured to *not* fsync() on all Append() calls
I20260806 21:07:01.095497 4309 tablet_bootstrap.cc:492] T 0913d0f9b1d342b580425f5a4e4b3342 P 0b706acb694f4024acecc217a6c7a3b5: No bootstrap required, opened a new log
I20260806 21:07:01.095707 4309 ts_tablet_manager.cc:1397] T 0913d0f9b1d342b580425f5a4e4b3342 P 0b706acb694f4024acecc217a6c7a3b5: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 21:07:01.096635 4310 tablet_bootstrap.cc:492] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8: No bootstrap required, opened a new log
I20260806 21:07:01.096843 4310 ts_tablet_manager.cc:1397] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:07:01.096980 4311 tablet_bootstrap.cc:654] T 0913d0f9b1d342b580425f5a4e4b3342 P ff6219ada81d4c209dc32acc02b15831: Neither blocks nor log segments found. Creating new log.
I20260806 21:07:01.097782 4311 log.cc:824] T 0913d0f9b1d342b580425f5a4e4b3342 P ff6219ada81d4c209dc32acc02b15831: Log is configured to *not* fsync() on all Append() calls
I20260806 21:07:01.099798 4311 tablet_bootstrap.cc:492] T 0913d0f9b1d342b580425f5a4e4b3342 P ff6219ada81d4c209dc32acc02b15831: No bootstrap required, opened a new log
I20260806 21:07:01.100065 4311 ts_tablet_manager.cc:1397] T 0913d0f9b1d342b580425f5a4e4b3342 P ff6219ada81d4c209dc32acc02b15831: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 21:07:01.100620 4309 raft_consensus.cc:348] T 0913d0f9b1d342b580425f5a4e4b3342 P 0b706acb694f4024acecc217a6c7a3b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } } peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } }
I20260806 21:07:01.101016 4309 raft_consensus.cc:374] T 0913d0f9b1d342b580425f5a4e4b3342 P 0b706acb694f4024acecc217a6c7a3b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:07:01.101099 4309 raft_consensus.cc:729] T 0913d0f9b1d342b580425f5a4e4b3342 P 0b706acb694f4024acecc217a6c7a3b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b706acb694f4024acecc217a6c7a3b5, State: Initialized, Role: FOLLOWER
I20260806 21:07:01.101622 4309 consensus_queue.cc:260] T 0913d0f9b1d342b580425f5a4e4b3342 P 0b706acb694f4024acecc217a6c7a3b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } } peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } }
I20260806 21:07:01.101991 4310 raft_consensus.cc:348] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } } peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } }
I20260806 21:07:01.102324 4310 raft_consensus.cc:374] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:07:01.102388 4310 raft_consensus.cc:729] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 871ebb27b0794ec0a464ba6e581a08b8, State: Initialized, Role: FOLLOWER
I20260806 21:07:01.102631 4281 heartbeater.cc:503] Master 127.27.225.254:33337 was elected leader, sending a full tablet report...
I20260806 21:07:01.102885 4310 consensus_queue.cc:260] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } } peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } }
I20260806 21:07:01.103026 4309 ts_tablet_manager.cc:1428] T 0913d0f9b1d342b580425f5a4e4b3342 P 0b706acb694f4024acecc217a6c7a3b5: Time spent starting tablet: real 0.007s user 0.009s sys 0.000s
I20260806 21:07:01.103848 4310 ts_tablet_manager.cc:1428] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260806 21:07:01.104277 3991 heartbeater.cc:503] Master 127.27.225.254:33337 was elected leader, sending a full tablet report...
I20260806 21:07:01.104303 4311 raft_consensus.cc:348] T 0913d0f9b1d342b580425f5a4e4b3342 P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } } peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } }
I20260806 21:07:01.104578 4311 raft_consensus.cc:374] T 0913d0f9b1d342b580425f5a4e4b3342 P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:07:01.104662 4311 raft_consensus.cc:729] T 0913d0f9b1d342b580425f5a4e4b3342 P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff6219ada81d4c209dc32acc02b15831, State: Initialized, Role: FOLLOWER
I20260806 21:07:01.105295 4311 consensus_queue.cc:260] T 0913d0f9b1d342b580425f5a4e4b3342 P ff6219ada81d4c209dc32acc02b15831 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } } peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } }
I20260806 21:07:01.106709 4311 ts_tablet_manager.cc:1428] T 0913d0f9b1d342b580425f5a4e4b3342 P ff6219ada81d4c209dc32acc02b15831: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 21:07:01.107093 4133 heartbeater.cc:503] Master 127.27.225.254:33337 was elected leader, sending a full tablet report...
I20260806 21:07:01.121181 4316 raft_consensus.cc:482] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:07:01.121429 4316 raft_consensus.cc:504] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } } peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } }
I20260806 21:07:01.122745 4316 leader_election.cc:288] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b706acb694f4024acecc217a6c7a3b5 (127.27.225.195:44843), ff6219ada81d4c209dc32acc02b15831 (127.27.225.194:35821)
I20260806 21:07:01.127604 4306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.123150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:37209 (local address 127.27.225.195:44843)
0806 21:07:01.123313 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:01.123318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:01.123490 (+ 172us) server_negotiation.cc:407] Connection header received
0806 21:07:01.123843 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:01.123850 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:01.123925 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:01.124023 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:07:01.124397 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.125203 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.126563 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.126996 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.127134 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.127191 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:01.127375 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 21:07:01.127466 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 21:07:01.127620 4308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.123685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:39069 (local address 127.27.225.194:35821)
0806 21:07:01.123830 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:01.123835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:01.123926 (+ 91us) server_negotiation.cc:407] Connection header received
0806 21:07:01.124010 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:01.124015 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:01.124059 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:01.124134 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:07:01.124397 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.125244 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.126610 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.127029 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.127166 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.127230 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:01.127411 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 21:07:01.127489 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 21:07:01.127648 4318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.123030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:44843 (local address 127.27.225.193:37209)
0806 21:07:01.123375 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:01.123393 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:01.123518 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:01.124062 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:01.124064 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:01.124076 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:07:01.124263 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:01.124271 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.125338 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.125342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:01.126428 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:01.126440 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.127129 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.127133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:01.127237 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.127268 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 21:07:01.127359 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 21:07:01.127437 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":114,"thread_start_us":178,"threads_started":1}
I20260806 21:07:01.127655 4319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.123523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:35821 (local address 127.27.225.193:39069)
0806 21:07:01.123843 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:01.123855 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:01.123934 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:01.124150 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:01.124152 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:01.124163 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:07:01.124288 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:01.124296 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.125379 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.125382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:01.126489 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:01.126502 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.127159 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.127163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:01.127272 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.127295 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 21:07:01.127392 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 21:07:01.127470 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":6,"thread_start_us":125,"threads_started":1}
I20260806 21:07:01.128363 4086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0913d0f9b1d342b580425f5a4e4b3342" candidate_uuid: "871ebb27b0794ec0a464ba6e581a08b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff6219ada81d4c209dc32acc02b15831" is_pre_election: true
I20260806 21:07:01.128361 4229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0913d0f9b1d342b580425f5a4e4b3342" candidate_uuid: "871ebb27b0794ec0a464ba6e581a08b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b706acb694f4024acecc217a6c7a3b5" is_pre_election: true
I20260806 21:07:01.128702 4086 raft_consensus.cc:2454] T 0913d0f9b1d342b580425f5a4e4b3342 P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 871ebb27b0794ec0a464ba6e581a08b8 in term 0.
I20260806 21:07:01.128712 4229 raft_consensus.cc:2454] T 0913d0f9b1d342b580425f5a4e4b3342 P 0b706acb694f4024acecc217a6c7a3b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 871ebb27b0794ec0a464ba6e581a08b8 in term 0.
I20260806 21:07:01.129345 3874 leader_election.cc:302] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 871ebb27b0794ec0a464ba6e581a08b8, ff6219ada81d4c209dc32acc02b15831; no voters:
I20260806 21:07:01.129627 4316 raft_consensus.cc:2781] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:07:01.129745 4316 raft_consensus.cc:482] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:07:01.129807 4316 raft_consensus.cc:3037] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:07:01.130916 4316 raft_consensus.cc:504] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } } peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } }
I20260806 21:07:01.131289 4316 leader_election.cc:288] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8 [CANDIDATE]: Term 1 election: Requested vote from peers 0b706acb694f4024acecc217a6c7a3b5 (127.27.225.195:44843), ff6219ada81d4c209dc32acc02b15831 (127.27.225.194:35821)
I20260806 21:07:01.131862 4086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0913d0f9b1d342b580425f5a4e4b3342" candidate_uuid: "871ebb27b0794ec0a464ba6e581a08b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff6219ada81d4c209dc32acc02b15831"
I20260806 21:07:01.131847 4229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0913d0f9b1d342b580425f5a4e4b3342" candidate_uuid: "871ebb27b0794ec0a464ba6e581a08b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b706acb694f4024acecc217a6c7a3b5"
I20260806 21:07:01.132001 4229 raft_consensus.cc:3037] T 0913d0f9b1d342b580425f5a4e4b3342 P 0b706acb694f4024acecc217a6c7a3b5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:07:01.132001 4086 raft_consensus.cc:3037] T 0913d0f9b1d342b580425f5a4e4b3342 P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:07:01.133430 4086 raft_consensus.cc:2454] T 0913d0f9b1d342b580425f5a4e4b3342 P ff6219ada81d4c209dc32acc02b15831 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 871ebb27b0794ec0a464ba6e581a08b8 in term 1.
I20260806 21:07:01.133430 4229 raft_consensus.cc:2454] T 0913d0f9b1d342b580425f5a4e4b3342 P 0b706acb694f4024acecc217a6c7a3b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 871ebb27b0794ec0a464ba6e581a08b8 in term 1.
I20260806 21:07:01.133800 3874 leader_election.cc:302] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b706acb694f4024acecc217a6c7a3b5, 871ebb27b0794ec0a464ba6e581a08b8; no voters:
I20260806 21:07:01.134038 4316 raft_consensus.cc:2781] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:07:01.134326 4316 raft_consensus.cc:686] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8 [term 1 LEADER]: Becoming Leader. State: Replica: 871ebb27b0794ec0a464ba6e581a08b8, State: Running, Role: LEADER
I20260806 21:07:01.134699 4316 consensus_queue.cc:237] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } } peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } }
I20260806 21:07:01.138406 3784 catalog_manager.cc:5495] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 871ebb27b0794ec0a464ba6e581a08b8 (127.27.225.193). New cstate: current_term: 1 leader_uuid: "871ebb27b0794ec0a464ba6e581a08b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } health_report { overall_health: UNKNOWN } } }
I20260806 21:07:01.163007 4305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.159248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51562 (local address 127.27.225.193:32999)
0806 21:07:01.159391 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:01.159396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:01.159413 (+ 17us) server_negotiation.cc:407] Connection header received
0806 21:07:01.159530 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:01.159532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:01.159588 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:01.159709 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:07:01.159952 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.160484 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.161357 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.161575 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.161681 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.162160 (+ 479us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:07:01.162600 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:07:01.162705 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:01.162836 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 21:07:01.162896 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 21:07:01.169219 4229 raft_consensus.cc:1261] T 0913d0f9b1d342b580425f5a4e4b3342 P 0b706acb694f4024acecc217a6c7a3b5 [term 1 FOLLOWER]: Refusing update from remote peer 871ebb27b0794ec0a464ba6e581a08b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:07:01.169298 4086 raft_consensus.cc:1261] T 0913d0f9b1d342b580425f5a4e4b3342 P ff6219ada81d4c209dc32acc02b15831 [term 1 FOLLOWER]: Refusing update from remote peer 871ebb27b0794ec0a464ba6e581a08b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:07:01.170125 4316 consensus_queue.cc:1035] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:07:01.170418 4320 consensus_queue.cc:1035] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:07:01.179754 4323 mvcc.cc:205] Tried to move back new op lower bound from 7315662525100810240 to 7315662524970758144. Current Snapshot: MvccSnapshot[applied={T|T < 7315662525100810240 or (T in {7315662525100810240})}]
I20260806 21:07:01.179981 4325 mvcc.cc:205] Tried to move back new op lower bound from 7315662525100810240 to 7315662524970758144. Current Snapshot: MvccSnapshot[applied={T|T < 7315662525100810240}]
I20260806 21:07:01.180320 4324 mvcc.cc:205] Tried to move back new op lower bound from 7315662525100810240 to 7315662524970758144. Current Snapshot: MvccSnapshot[applied={T|T < 7315662525100810240}]
I20260806 21:07:01.187469 3783 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:07:01.187661 3783 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:07:01.190382 3783 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f: Sending DeleteTablet for 3 replicas of tablet 0913d0f9b1d342b580425f5a4e4b3342
I20260806 21:07:01.191128 3924 tablet_service.cc:1514] Processing DeleteTablet for tablet 0913d0f9b1d342b580425f5a4e4b3342 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:07:01 UTC) from {username='oryx', principal='oryx/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:51548
I20260806 21:07:01.191136 4206 tablet_service.cc:1514] Processing DeleteTablet for tablet 0913d0f9b1d342b580425f5a4e4b3342 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:07:01 UTC) from {username='oryx', principal='oryx/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:45390
I20260806 21:07:01.191861 4332 tablet_replica.cc:330] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8: stopping tablet replica
I20260806 21:07:01.192176 4332 raft_consensus.cc:2229] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:07:01.192340 4066 tablet_service.cc:1514] Processing DeleteTablet for tablet 0913d0f9b1d342b580425f5a4e4b3342 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:07:01 UTC) from {username='oryx', principal='oryx/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:57138
I20260806 21:07:01.192548 4332 raft_consensus.cc:2258] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:07:01.193514 4334 tablet_replica.cc:330] T 0913d0f9b1d342b580425f5a4e4b3342 P ff6219ada81d4c209dc32acc02b15831: stopping tablet replica
I20260806 21:07:01.193619 3783 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51836:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 21:07:01.194046 4334 raft_consensus.cc:2229] T 0913d0f9b1d342b580425f5a4e4b3342 P ff6219ada81d4c209dc32acc02b15831 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 21:07:01.194095 3783 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:07:01.194409 4334 raft_consensus.cc:2258] T 0913d0f9b1d342b580425f5a4e4b3342 P ff6219ada81d4c209dc32acc02b15831 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:07:01.195749 4332 ts_tablet_manager.cc:1905] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:07:01.198740 4332 ts_tablet_manager.cc:1918] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:07:01.198910 4332 log.cc:1196] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-0/wal/wals/0913d0f9b1d342b580425f5a4e4b3342
I20260806 21:07:01.199497 3924 tablet_service.cc:1467] Processing CreateTablet for tablet df52a7d2679d42389ca25f913ce20ce8 (DEFAULT_TABLE table=test-table [id=8fb159bc4645487fa58b6dde884ae762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:07:01.199726 3924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df52a7d2679d42389ca25f913ce20ce8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:07:01.200237 4206 tablet_service.cc:1467] Processing CreateTablet for tablet df52a7d2679d42389ca25f913ce20ce8 (DEFAULT_TABLE table=test-table [id=8fb159bc4645487fa58b6dde884ae762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:07:01.200497 4206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df52a7d2679d42389ca25f913ce20ce8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:07:01.201972 4310 tablet_bootstrap.cc:492] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8: Bootstrap starting.
I20260806 21:07:01.202916 4310 tablet_bootstrap.cc:654] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8: Neither blocks nor log segments found. Creating new log.
I20260806 21:07:01.203392 4066 tablet_service.cc:1467] Processing CreateTablet for tablet df52a7d2679d42389ca25f913ce20ce8 (DEFAULT_TABLE table=test-table [id=8fb159bc4645487fa58b6dde884ae762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:07:01.203500 4309 tablet_bootstrap.cc:492] T df52a7d2679d42389ca25f913ce20ce8 P 0b706acb694f4024acecc217a6c7a3b5: Bootstrap starting.
I20260806 21:07:01.203652 4066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df52a7d2679d42389ca25f913ce20ce8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:07:01.204234 4332 ts_tablet_manager.cc:1939] T 0913d0f9b1d342b580425f5a4e4b3342 P 871ebb27b0794ec0a464ba6e581a08b8: Deleting consensus metadata
I20260806 21:07:01.204559 4309 tablet_bootstrap.cc:654] T df52a7d2679d42389ca25f913ce20ce8 P 0b706acb694f4024acecc217a6c7a3b5: Neither blocks nor log segments found. Creating new log.
I20260806 21:07:01.205969 4309 tablet_bootstrap.cc:492] T df52a7d2679d42389ca25f913ce20ce8 P 0b706acb694f4024acecc217a6c7a3b5: No bootstrap required, opened a new log
I20260806 21:07:01.206069 4309 ts_tablet_manager.cc:1397] T df52a7d2679d42389ca25f913ce20ce8 P 0b706acb694f4024acecc217a6c7a3b5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:07:01.206112 4311 tablet_bootstrap.cc:492] T df52a7d2679d42389ca25f913ce20ce8 P ff6219ada81d4c209dc32acc02b15831: Bootstrap starting.
I20260806 21:07:01.206255 3769 catalog_manager.cc:4841] TS 871ebb27b0794ec0a464ba6e581a08b8 (127.27.225.193:32999): tablet 0913d0f9b1d342b580425f5a4e4b3342 (table test-table [id=12a9c35a87a04206a9b6e6c576b52432]) successfully deleted
I20260806 21:07:01.206459 4310 tablet_bootstrap.cc:492] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8: No bootstrap required, opened a new log
I20260806 21:07:01.206533 4310 ts_tablet_manager.cc:1397] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 21:07:01.206566 4309 raft_consensus.cc:348] T df52a7d2679d42389ca25f913ce20ce8 P 0b706acb694f4024acecc217a6c7a3b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.206682 4309 raft_consensus.cc:374] T df52a7d2679d42389ca25f913ce20ce8 P 0b706acb694f4024acecc217a6c7a3b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:07:01.206714 4309 raft_consensus.cc:729] T df52a7d2679d42389ca25f913ce20ce8 P 0b706acb694f4024acecc217a6c7a3b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b706acb694f4024acecc217a6c7a3b5, State: Initialized, Role: FOLLOWER
I20260806 21:07:01.206854 4309 consensus_queue.cc:260] T df52a7d2679d42389ca25f913ce20ce8 P 0b706acb694f4024acecc217a6c7a3b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.206995 4310 raft_consensus.cc:348] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.207093 4310 raft_consensus.cc:374] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:07:01.207123 4310 raft_consensus.cc:729] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 871ebb27b0794ec0a464ba6e581a08b8, State: Initialized, Role: FOLLOWER
I20260806 21:07:01.207129 4309 ts_tablet_manager.cc:1428] T df52a7d2679d42389ca25f913ce20ce8 P 0b706acb694f4024acecc217a6c7a3b5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:07:01.207130 4311 tablet_bootstrap.cc:654] T df52a7d2679d42389ca25f913ce20ce8 P ff6219ada81d4c209dc32acc02b15831: Neither blocks nor log segments found. Creating new log.
I20260806 21:07:01.207248 4310 consensus_queue.cc:260] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.207515 4310 ts_tablet_manager.cc:1428] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:07:01.208124 4333 tablet_replica.cc:330] T 0913d0f9b1d342b580425f5a4e4b3342 P 0b706acb694f4024acecc217a6c7a3b5: stopping tablet replica
I20260806 21:07:01.208520 4333 raft_consensus.cc:2229] T 0913d0f9b1d342b580425f5a4e4b3342 P 0b706acb694f4024acecc217a6c7a3b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:07:01.208765 4311 tablet_bootstrap.cc:492] T df52a7d2679d42389ca25f913ce20ce8 P ff6219ada81d4c209dc32acc02b15831: No bootstrap required, opened a new log
I20260806 21:07:01.208869 4311 ts_tablet_manager.cc:1397] T df52a7d2679d42389ca25f913ce20ce8 P ff6219ada81d4c209dc32acc02b15831: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:07:01.208884 4334 ts_tablet_manager.cc:1905] T 0913d0f9b1d342b580425f5a4e4b3342 P ff6219ada81d4c209dc32acc02b15831: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:07:01.208892 4333 raft_consensus.cc:2258] T 0913d0f9b1d342b580425f5a4e4b3342 P 0b706acb694f4024acecc217a6c7a3b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:07:01.209333 4311 raft_consensus.cc:348] T df52a7d2679d42389ca25f913ce20ce8 P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.209450 4311 raft_consensus.cc:374] T df52a7d2679d42389ca25f913ce20ce8 P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:07:01.209489 4311 raft_consensus.cc:729] T df52a7d2679d42389ca25f913ce20ce8 P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff6219ada81d4c209dc32acc02b15831, State: Initialized, Role: FOLLOWER
I20260806 21:07:01.209620 4311 consensus_queue.cc:260] T df52a7d2679d42389ca25f913ce20ce8 P ff6219ada81d4c209dc32acc02b15831 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.211480 4311 ts_tablet_manager.cc:1428] T df52a7d2679d42389ca25f913ce20ce8 P ff6219ada81d4c209dc32acc02b15831: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260806 21:07:01.211925 4334 ts_tablet_manager.cc:1918] T 0913d0f9b1d342b580425f5a4e4b3342 P ff6219ada81d4c209dc32acc02b15831: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:07:01.212041 4334 log.cc:1196] T 0913d0f9b1d342b580425f5a4e4b3342 P ff6219ada81d4c209dc32acc02b15831: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-1/wal/wals/0913d0f9b1d342b580425f5a4e4b3342
I20260806 21:07:01.212445 4334 ts_tablet_manager.cc:1939] T 0913d0f9b1d342b580425f5a4e4b3342 P ff6219ada81d4c209dc32acc02b15831: Deleting consensus metadata
I20260806 21:07:01.213367 4333 ts_tablet_manager.cc:1905] T 0913d0f9b1d342b580425f5a4e4b3342 P 0b706acb694f4024acecc217a6c7a3b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:07:01.213804 3767 catalog_manager.cc:4841] TS ff6219ada81d4c209dc32acc02b15831 (127.27.225.194:35821): tablet 0913d0f9b1d342b580425f5a4e4b3342 (table test-table [id=12a9c35a87a04206a9b6e6c576b52432]) successfully deleted
I20260806 21:07:01.215669 4333 ts_tablet_manager.cc:1918] T 0913d0f9b1d342b580425f5a4e4b3342 P 0b706acb694f4024acecc217a6c7a3b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:07:01.215795 4333 log.cc:1196] T 0913d0f9b1d342b580425f5a4e4b3342 P 0b706acb694f4024acecc217a6c7a3b5: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/0913d0f9b1d342b580425f5a4e4b3342
I20260806 21:07:01.216133 4333 ts_tablet_manager.cc:1939] T 0913d0f9b1d342b580425f5a4e4b3342 P 0b706acb694f4024acecc217a6c7a3b5: Deleting consensus metadata
I20260806 21:07:01.217432 3767 catalog_manager.cc:4841] TS 0b706acb694f4024acecc217a6c7a3b5 (127.27.225.195:44843): tablet 0913d0f9b1d342b580425f5a4e4b3342 (table test-table [id=12a9c35a87a04206a9b6e6c576b52432]) successfully deleted
W20260806 21:07:01.222656 3992 tablet.cc:2369] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 21:07:01.243031 4282 tablet.cc:2369] T df52a7d2679d42389ca25f913ce20ce8 P 0b706acb694f4024acecc217a6c7a3b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:07:01.294176 4320 raft_consensus.cc:482] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:07:01.294310 4320 raft_consensus.cc:504] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.294796 4320 leader_election.cc:288] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff6219ada81d4c209dc32acc02b15831 (127.27.225.194:35821), 0b706acb694f4024acecc217a6c7a3b5 (127.27.225.195:44843)
I20260806 21:07:01.295249 4086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df52a7d2679d42389ca25f913ce20ce8" candidate_uuid: "871ebb27b0794ec0a464ba6e581a08b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff6219ada81d4c209dc32acc02b15831" is_pre_election: true
I20260806 21:07:01.295357 4229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df52a7d2679d42389ca25f913ce20ce8" candidate_uuid: "871ebb27b0794ec0a464ba6e581a08b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b706acb694f4024acecc217a6c7a3b5" is_pre_election: true
I20260806 21:07:01.295424 4086 raft_consensus.cc:2454] T df52a7d2679d42389ca25f913ce20ce8 P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 871ebb27b0794ec0a464ba6e581a08b8 in term 0.
I20260806 21:07:01.295536 4229 raft_consensus.cc:2454] T df52a7d2679d42389ca25f913ce20ce8 P 0b706acb694f4024acecc217a6c7a3b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 871ebb27b0794ec0a464ba6e581a08b8 in term 0.
I20260806 21:07:01.295753 3874 leader_election.cc:302] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 871ebb27b0794ec0a464ba6e581a08b8, ff6219ada81d4c209dc32acc02b15831; no voters:
I20260806 21:07:01.295976 4320 raft_consensus.cc:2781] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:07:01.296052 4320 raft_consensus.cc:482] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:07:01.296093 4320 raft_consensus.cc:3037] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:07:01.298018 4320 raft_consensus.cc:504] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.298429 4320 leader_election.cc:288] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8 [CANDIDATE]: Term 1 election: Requested vote from peers ff6219ada81d4c209dc32acc02b15831 (127.27.225.194:35821), 0b706acb694f4024acecc217a6c7a3b5 (127.27.225.195:44843)
I20260806 21:07:01.298991 4086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df52a7d2679d42389ca25f913ce20ce8" candidate_uuid: "871ebb27b0794ec0a464ba6e581a08b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff6219ada81d4c209dc32acc02b15831"
I20260806 21:07:01.299057 4229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df52a7d2679d42389ca25f913ce20ce8" candidate_uuid: "871ebb27b0794ec0a464ba6e581a08b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b706acb694f4024acecc217a6c7a3b5"
I20260806 21:07:01.299158 4086 raft_consensus.cc:3037] T df52a7d2679d42389ca25f913ce20ce8 P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:07:01.299187 4229 raft_consensus.cc:3037] T df52a7d2679d42389ca25f913ce20ce8 P 0b706acb694f4024acecc217a6c7a3b5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:07:01.300486 4086 raft_consensus.cc:2454] T df52a7d2679d42389ca25f913ce20ce8 P ff6219ada81d4c209dc32acc02b15831 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 871ebb27b0794ec0a464ba6e581a08b8 in term 1.
I20260806 21:07:01.300485 4229 raft_consensus.cc:2454] T df52a7d2679d42389ca25f913ce20ce8 P 0b706acb694f4024acecc217a6c7a3b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 871ebb27b0794ec0a464ba6e581a08b8 in term 1.
I20260806 21:07:01.300869 3874 leader_election.cc:302] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b706acb694f4024acecc217a6c7a3b5, 871ebb27b0794ec0a464ba6e581a08b8; no voters:
I20260806 21:07:01.301183 4320 raft_consensus.cc:2781] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:07:01.301343 4320 raft_consensus.cc:686] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8 [term 1 LEADER]: Becoming Leader. State: Replica: 871ebb27b0794ec0a464ba6e581a08b8, State: Running, Role: LEADER
I20260806 21:07:01.301513 4320 consensus_queue.cc:237] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.303447 3784 catalog_manager.cc:5495] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 871ebb27b0794ec0a464ba6e581a08b8 (127.27.225.193). New cstate: current_term: 1 leader_uuid: "871ebb27b0794ec0a464ba6e581a08b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } health_report { overall_health: UNKNOWN } } }
Aug 06 21:07:01 dist-test-slave-69rg krb5kdc[3735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050420, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.254@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM
I20260806 21:07:01.353574 4344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.344092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:33337 (local address 127.0.0.1:51852)
0806 21:07:01.344416 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:01.344433 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:01.344551 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:01.344853 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:01.344856 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:01.345289 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:01.345395 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:01.345402 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.346496 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.346499 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:01.347055 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:01.347063 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.347642 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.347645 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:01.347709 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.348310 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:01.348322 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:01.350074 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:07:01.352015 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:01.352024 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:01.352030 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:01.352264 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:01.352534 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:01.352536 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:01.352538 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:01.352689 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:01.353092 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:07:01.353098 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:07:01.353216 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 21:07:01.353312 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 21:07:01.353433 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":138,"threads_started":1}
I20260806 21:07:01.353801 3835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.344146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51852 (local address 127.27.225.254:33337)
0806 21:07:01.344342 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:01.344349 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:01.344530 (+ 181us) server_negotiation.cc:407] Connection header received
0806 21:07:01.344628 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:01.344631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:01.344686 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:01.344800 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:01.345549 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.346377 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.347172 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.347517 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.347617 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.350229 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:01.350233 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:01.350236 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:07:01.350263 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:01.351911 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:01.352358 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:01.352361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:01.352363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:01.352448 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:01.352759 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:01.352763 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:01.352764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:01.352993 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:01.353088 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:01.353384 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 21:07:01.353504 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 21:07:01.358175 3784 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:51852:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 21:07:01.359002 3784 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:07:01.364262 3924 tablet_service.cc:1467] Processing CreateTablet for tablet 386babc6f2214c3393672f814ff653cd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=333abb0e015541e9bc1c07fe48f7310a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 21:07:01.364262 4066 tablet_service.cc:1467] Processing CreateTablet for tablet 386babc6f2214c3393672f814ff653cd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=333abb0e015541e9bc1c07fe48f7310a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 21:07:01.364403 4206 tablet_service.cc:1467] Processing CreateTablet for tablet 386babc6f2214c3393672f814ff653cd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=333abb0e015541e9bc1c07fe48f7310a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 21:07:01.364678 3924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 386babc6f2214c3393672f814ff653cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:07:01.364678 4066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 386babc6f2214c3393672f814ff653cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:07:01.364840 4206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 386babc6f2214c3393672f814ff653cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:07:01.367489 4311 tablet_bootstrap.cc:492] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831: Bootstrap starting.
I20260806 21:07:01.367520 4310 tablet_bootstrap.cc:492] T 386babc6f2214c3393672f814ff653cd P 871ebb27b0794ec0a464ba6e581a08b8: Bootstrap starting.
I20260806 21:07:01.368423 4311 tablet_bootstrap.cc:654] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831: Neither blocks nor log segments found. Creating new log.
I20260806 21:07:01.368499 4310 tablet_bootstrap.cc:654] T 386babc6f2214c3393672f814ff653cd P 871ebb27b0794ec0a464ba6e581a08b8: Neither blocks nor log segments found. Creating new log.
I20260806 21:07:01.369130 4309 tablet_bootstrap.cc:492] T 386babc6f2214c3393672f814ff653cd P 0b706acb694f4024acecc217a6c7a3b5: Bootstrap starting.
I20260806 21:07:01.369657 4311 tablet_bootstrap.cc:492] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831: No bootstrap required, opened a new log
I20260806 21:07:01.369738 4311 ts_tablet_manager.cc:1397] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 21:07:01.370100 4309 tablet_bootstrap.cc:654] T 386babc6f2214c3393672f814ff653cd P 0b706acb694f4024acecc217a6c7a3b5: Neither blocks nor log segments found. Creating new log.
I20260806 21:07:01.370357 4311 raft_consensus.cc:348] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.370503 4311 raft_consensus.cc:374] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:07:01.370551 4311 raft_consensus.cc:729] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff6219ada81d4c209dc32acc02b15831, State: Initialized, Role: FOLLOWER
I20260806 21:07:01.370708 4311 consensus_queue.cc:260] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.371325 4310 tablet_bootstrap.cc:492] T 386babc6f2214c3393672f814ff653cd P 871ebb27b0794ec0a464ba6e581a08b8: No bootstrap required, opened a new log
I20260806 21:07:01.371390 4309 tablet_bootstrap.cc:492] T 386babc6f2214c3393672f814ff653cd P 0b706acb694f4024acecc217a6c7a3b5: No bootstrap required, opened a new log
I20260806 21:07:01.371443 4310 ts_tablet_manager.cc:1397] T 386babc6f2214c3393672f814ff653cd P 871ebb27b0794ec0a464ba6e581a08b8: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260806 21:07:01.371497 4309 ts_tablet_manager.cc:1397] T 386babc6f2214c3393672f814ff653cd P 0b706acb694f4024acecc217a6c7a3b5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 21:07:01.371860 4310 raft_consensus.cc:348] T 386babc6f2214c3393672f814ff653cd P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.371870 4309 raft_consensus.cc:348] T 386babc6f2214c3393672f814ff653cd P 0b706acb694f4024acecc217a6c7a3b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.371981 4309 raft_consensus.cc:374] T 386babc6f2214c3393672f814ff653cd P 0b706acb694f4024acecc217a6c7a3b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:07:01.371986 4310 raft_consensus.cc:374] T 386babc6f2214c3393672f814ff653cd P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:07:01.372030 4309 raft_consensus.cc:729] T 386babc6f2214c3393672f814ff653cd P 0b706acb694f4024acecc217a6c7a3b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b706acb694f4024acecc217a6c7a3b5, State: Initialized, Role: FOLLOWER
I20260806 21:07:01.372040 4310 raft_consensus.cc:729] T 386babc6f2214c3393672f814ff653cd P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 871ebb27b0794ec0a464ba6e581a08b8, State: Initialized, Role: FOLLOWER
I20260806 21:07:01.372165 4309 consensus_queue.cc:260] T 386babc6f2214c3393672f814ff653cd P 0b706acb694f4024acecc217a6c7a3b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.372197 4310 consensus_queue.cc:260] T 386babc6f2214c3393672f814ff653cd P 871ebb27b0794ec0a464ba6e581a08b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.372479 4309 ts_tablet_manager.cc:1428] T 386babc6f2214c3393672f814ff653cd P 0b706acb694f4024acecc217a6c7a3b5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 21:07:01.372428 4315 tablet_replica.cc:439] T 386babc6f2214c3393672f814ff653cd P 0b706acb694f4024acecc217a6c7a3b5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } } }
I20260806 21:07:01.372450 4320 tablet_replica.cc:439] T 386babc6f2214c3393672f814ff653cd P 871ebb27b0794ec0a464ba6e581a08b8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } } }
I20260806 21:07:01.372570 4315 tablet_replica.cc:442] T 386babc6f2214c3393672f814ff653cd P 0b706acb694f4024acecc217a6c7a3b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:07:01.372581 4320 tablet_replica.cc:442] T 386babc6f2214c3393672f814ff653cd P 871ebb27b0794ec0a464ba6e581a08b8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:07:01.372818 4310 ts_tablet_manager.cc:1428] T 386babc6f2214c3393672f814ff653cd P 871ebb27b0794ec0a464ba6e581a08b8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:07:01.373086 4317 tablet_replica.cc:439] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } } }
I20260806 21:07:01.373238 4317 tablet_replica.cc:442] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:07:01.374114 4311 ts_tablet_manager.cc:1428] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
W20260806 21:07:01.458153 4134 tablet.cc:2369] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:07:01.532413 4317 raft_consensus.cc:482] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:07:01.532574 4317 raft_consensus.cc:504] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.533798 4317 leader_election.cc:288] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 871ebb27b0794ec0a464ba6e581a08b8 (127.27.225.193:32999), 0b706acb694f4024acecc217a6c7a3b5 (127.27.225.195:44843)
I20260806 21:07:01.538131 4306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.534219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:59091 (local address 127.27.225.195:44843)
0806 21:07:01.534372 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:01.534377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:01.534688 (+ 311us) server_negotiation.cc:407] Connection header received
0806 21:07:01.534959 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:01.534962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:01.535030 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:01.535109 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:07:01.535438 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.535917 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.537097 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.537495 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.537624 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.537689 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:01.537922 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 21:07:01.537995 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 21:07:01.538158 4349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.534158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:44843 (local address 127.27.225.194:59091)
0806 21:07:01.534531 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:01.534545 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:01.534824 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:01.535153 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:01.535156 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:01.535199 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:07:01.535323 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:01.535332 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.536056 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.536059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:01.536972 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:01.536981 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.537623 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.537626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:01.537741 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.537768 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 21:07:01.537904 (+ 136us) client_negotiation.cc:240] Negotiation successful
0806 21:07:01.537977 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":45,"thread_start_us":67,"threads_started":1}
I20260806 21:07:01.538285 4305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.534277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:59027 (local address 127.27.225.193:32999)
0806 21:07:01.534464 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:01.534471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:01.534652 (+ 181us) server_negotiation.cc:407] Connection header received
0806 21:07:01.534877 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:01.534883 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:01.534947 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:01.535017 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:07:01.535352 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.535899 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.537132 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.537572 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.537649 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.537690 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:01.538050 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 21:07:01.538145 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 21:07:01.538511 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.534109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:32999 (local address 127.27.225.194:59027)
0806 21:07:01.534531 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:01.534546 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:01.534713 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:01.535058 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:01.535064 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:01.535086 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:07:01.535232 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:01.535239 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.536016 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.536020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:01.536936 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:01.536945 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.537771 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.537773 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:01.537930 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.537947 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 21:07:01.538030 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 21:07:01.538116 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":86,"thread_start_us":102,"threads_started":1}
I20260806 21:07:01.538751 4229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386babc6f2214c3393672f814ff653cd" candidate_uuid: "ff6219ada81d4c209dc32acc02b15831" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b706acb694f4024acecc217a6c7a3b5" is_pre_election: true
I20260806 21:07:01.538945 4229 raft_consensus.cc:2454] T 386babc6f2214c3393672f814ff653cd P 0b706acb694f4024acecc217a6c7a3b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff6219ada81d4c209dc32acc02b15831 in term 0.
I20260806 21:07:01.539325 4017 leader_election.cc:302] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b706acb694f4024acecc217a6c7a3b5, ff6219ada81d4c209dc32acc02b15831; no voters:
I20260806 21:07:01.539306 3944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386babc6f2214c3393672f814ff653cd" candidate_uuid: "ff6219ada81d4c209dc32acc02b15831" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "871ebb27b0794ec0a464ba6e581a08b8" is_pre_election: true
I20260806 21:07:01.539577 3944 raft_consensus.cc:2454] T 386babc6f2214c3393672f814ff653cd P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff6219ada81d4c209dc32acc02b15831 in term 0.
I20260806 21:07:01.539610 4317 raft_consensus.cc:2781] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:07:01.539669 4317 raft_consensus.cc:482] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:07:01.539708 4317 raft_consensus.cc:3037] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:07:01.540926 4317 raft_consensus.cc:504] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.541297 4317 leader_election.cc:288] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831 [CANDIDATE]: Term 1 election: Requested vote from peers 871ebb27b0794ec0a464ba6e581a08b8 (127.27.225.193:32999), 0b706acb694f4024acecc217a6c7a3b5 (127.27.225.195:44843)
I20260806 21:07:01.541635 3944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386babc6f2214c3393672f814ff653cd" candidate_uuid: "ff6219ada81d4c209dc32acc02b15831" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "871ebb27b0794ec0a464ba6e581a08b8"
I20260806 21:07:01.541689 4229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386babc6f2214c3393672f814ff653cd" candidate_uuid: "ff6219ada81d4c209dc32acc02b15831" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b706acb694f4024acecc217a6c7a3b5"
I20260806 21:07:01.541774 3944 raft_consensus.cc:3037] T 386babc6f2214c3393672f814ff653cd P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:07:01.541882 4229 raft_consensus.cc:3037] T 386babc6f2214c3393672f814ff653cd P 0b706acb694f4024acecc217a6c7a3b5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:07:01.543064 3944 raft_consensus.cc:2454] T 386babc6f2214c3393672f814ff653cd P 871ebb27b0794ec0a464ba6e581a08b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff6219ada81d4c209dc32acc02b15831 in term 1.
I20260806 21:07:01.543059 4229 raft_consensus.cc:2454] T 386babc6f2214c3393672f814ff653cd P 0b706acb694f4024acecc217a6c7a3b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff6219ada81d4c209dc32acc02b15831 in term 1.
I20260806 21:07:01.543380 4019 leader_election.cc:302] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 871ebb27b0794ec0a464ba6e581a08b8, ff6219ada81d4c209dc32acc02b15831; no voters:
I20260806 21:07:01.543571 4317 raft_consensus.cc:2781] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:07:01.543795 4317 raft_consensus.cc:686] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831 [term 1 LEADER]: Becoming Leader. State: Replica: ff6219ada81d4c209dc32acc02b15831, State: Running, Role: LEADER
I20260806 21:07:01.544070 4317 consensus_queue.cc:237] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.545768 4350 tablet_replica.cc:439] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831: TxnStatusTablet state changed. Reason: New leader ff6219ada81d4c209dc32acc02b15831. Latest consensus state: current_term: 1 leader_uuid: "ff6219ada81d4c209dc32acc02b15831" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } } }
I20260806 21:07:01.545917 4350 tablet_replica.cc:442] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831: This TxnStatusTablet replica's current role is: LEADER
I20260806 21:07:01.546239 4352 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 21:07:01.546402 4352 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 21:07:01.546780 3782 catalog_manager.cc:5495] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff6219ada81d4c209dc32acc02b15831 (127.27.225.194). New cstate: current_term: 1 leader_uuid: "ff6219ada81d4c209dc32acc02b15831" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } health_report { overall_health: UNKNOWN } } }
I20260806 21:07:01.568836 4344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.564455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:35821 (local address 127.0.0.1:57142)
0806 21:07:01.564665 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:01.564685 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:01.564782 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:01.565214 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:01.565217 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:01.565227 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 21:07:01.565344 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:01.565350 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.566241 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.566244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:01.567218 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:01.567228 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.567660 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.567664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:01.567758 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.567814 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 21:07:01.568513 (+ 699us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 21:07:01.568529 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 21:07:01.568602 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 21:07:01.568689 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260806 21:07:01.568830 4308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.564584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57142 (local address 127.27.225.194:35821)
0806 21:07:01.564954 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:01.564957 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:01.564968 (+ 11us) server_negotiation.cc:407] Connection header received
0806 21:07:01.565030 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:01.565032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:01.565082 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:01.565156 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:07:01.565457 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.566124 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.567339 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.567551 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.567629 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.567972 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:07:01.568414 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:07:01.568504 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:01.568629 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 21:07:01.568682 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283}
I20260806 21:07:01.571914 4229 raft_consensus.cc:1261] T 386babc6f2214c3393672f814ff653cd P 0b706acb694f4024acecc217a6c7a3b5 [term 1 FOLLOWER]: Refusing update from remote peer ff6219ada81d4c209dc32acc02b15831: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:07:01.572108 3944 raft_consensus.cc:1261] T 386babc6f2214c3393672f814ff653cd P 871ebb27b0794ec0a464ba6e581a08b8 [term 1 FOLLOWER]: Refusing update from remote peer ff6219ada81d4c209dc32acc02b15831: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:07:01.572558 4317 consensus_queue.cc:1035] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:07:01.572727 4350 consensus_queue.cc:1035] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831 [LEADER]: Connected to new peer: Peer: permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:07:01.574226 4315 tablet_replica.cc:439] T 386babc6f2214c3393672f814ff653cd P 0b706acb694f4024acecc217a6c7a3b5: TxnStatusTablet state changed. Reason: New leader ff6219ada81d4c209dc32acc02b15831. Latest consensus state: current_term: 1 leader_uuid: "ff6219ada81d4c209dc32acc02b15831" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } } }
I20260806 21:07:01.574354 4315 tablet_replica.cc:442] T 386babc6f2214c3393672f814ff653cd P 0b706acb694f4024acecc217a6c7a3b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:07:01.575460 4320 tablet_replica.cc:439] T 386babc6f2214c3393672f814ff653cd P 871ebb27b0794ec0a464ba6e581a08b8: TxnStatusTablet state changed. Reason: New leader ff6219ada81d4c209dc32acc02b15831. Latest consensus state: current_term: 1 leader_uuid: "ff6219ada81d4c209dc32acc02b15831" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } } }
I20260806 21:07:01.575610 4320 tablet_replica.cc:442] T 386babc6f2214c3393672f814ff653cd P 871ebb27b0794ec0a464ba6e581a08b8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:07:01.577239 4350 tablet_replica.cc:439] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ff6219ada81d4c209dc32acc02b15831" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } } }
I20260806 21:07:01.577370 4350 tablet_replica.cc:442] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831: This TxnStatusTablet replica's current role is: LEADER
I20260806 21:07:01.578392 4320 tablet_replica.cc:439] T 386babc6f2214c3393672f814ff653cd P 871ebb27b0794ec0a464ba6e581a08b8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ff6219ada81d4c209dc32acc02b15831" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } } }
I20260806 21:07:01.578508 4320 tablet_replica.cc:442] T 386babc6f2214c3393672f814ff653cd P 871ebb27b0794ec0a464ba6e581a08b8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:07:01.581753 4317 tablet_replica.cc:439] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ff6219ada81d4c209dc32acc02b15831" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } } }
I20260806 21:07:01.581903 4317 tablet_replica.cc:442] T 386babc6f2214c3393672f814ff653cd P ff6219ada81d4c209dc32acc02b15831: This TxnStatusTablet replica's current role is: LEADER
I20260806 21:07:01.583843 4315 tablet_replica.cc:439] T 386babc6f2214c3393672f814ff653cd P 0b706acb694f4024acecc217a6c7a3b5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ff6219ada81d4c209dc32acc02b15831" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } } }
I20260806 21:07:01.583966 4315 tablet_replica.cc:442] T 386babc6f2214c3393672f814ff653cd P 0b706acb694f4024acecc217a6c7a3b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:07:01.593257 4360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.589652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:35821 (local address 127.27.225.193:46767)
0806 21:07:01.590077 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:01.590091 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:01.590219 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:01.590486 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:01.590489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:01.590498 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 21:07:01.590614 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:01.590620 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.591376 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.591382 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:01.592018 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:01.592027 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.592488 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.592491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:01.592587 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.592647 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 21:07:01.592961 (+ 314us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 21:07:01.592970 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 21:07:01.593038 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 21:07:01.593126 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":126,"threads_started":1}
I20260806 21:07:01.593237 4308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.589754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:46767 (local address 127.27.225.194:35821)
0806 21:07:01.589918 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:01.589923 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:01.590171 (+ 248us) server_negotiation.cc:407] Connection header received
0806 21:07:01.590336 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:01.590339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:01.590393 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:01.590474 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:07:01.590732 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.591261 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.592143 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.592372 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.592484 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.592808 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:07:01.592880 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:07:01.592956 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:01.593046 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 21:07:01.593094 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 21:07:01.600162 4086 raft_consensus.cc:1261] T df52a7d2679d42389ca25f913ce20ce8 P ff6219ada81d4c209dc32acc02b15831 [term 1 FOLLOWER]: Refusing update from remote peer 871ebb27b0794ec0a464ba6e581a08b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:07:01.600668 4320 consensus_queue.cc:1035] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:07:01.600802 4229 raft_consensus.cc:1261] T df52a7d2679d42389ca25f913ce20ce8 P 0b706acb694f4024acecc217a6c7a3b5 [term 1 FOLLOWER]: Refusing update from remote peer 871ebb27b0794ec0a464ba6e581a08b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:07:01.601253 4320 consensus_queue.cc:1035] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:07:01.625972 4364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.621792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:32999 (local address 127.27.225.194:49409)
0806 21:07:01.622202 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:01.622219 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:01.622316 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:01.622818 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:01.622821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:01.622830 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 21:07:01.622983 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:01.622990 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.623895 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.623900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:01.624667 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:01.624678 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.625166 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.625169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:01.625277 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.625322 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 21:07:01.625626 (+ 304us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 21:07:01.625636 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 21:07:01.625696 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 21:07:01.625780 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":125,"threads_started":1}
I20260806 21:07:01.625955 4305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.621924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:49409 (local address 127.27.225.193:32999)
0806 21:07:01.622068 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:01.622073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:01.622294 (+ 221us) server_negotiation.cc:407] Connection header received
0806 21:07:01.622389 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:01.622392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:01.622446 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:01.622544 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:07:01.623104 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.623784 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.624800 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.625043 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.625155 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.625462 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:07:01.625541 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:07:01.625622 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:01.625719 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 21:07:01.625777 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 21:07:01.667853 3782 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:07:01.668149 3782 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:07:01.670763 3782 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f: Sending DeleteTablet for 3 replicas of tablet df52a7d2679d42389ca25f913ce20ce8
I20260806 21:07:01.671482 4066 tablet_service.cc:1514] Processing DeleteTablet for tablet df52a7d2679d42389ca25f913ce20ce8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:07:01 UTC) from {username='oryx', principal='oryx/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:57138
I20260806 21:07:01.671627 4206 tablet_service.cc:1514] Processing DeleteTablet for tablet df52a7d2679d42389ca25f913ce20ce8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:07:01 UTC) from {username='oryx', principal='oryx/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:45390
I20260806 21:07:01.671651 3924 tablet_service.cc:1514] Processing DeleteTablet for tablet df52a7d2679d42389ca25f913ce20ce8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:07:01 UTC) from {username='oryx', principal='oryx/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:51548
I20260806 21:07:01.671727 4334 tablet_replica.cc:330] T df52a7d2679d42389ca25f913ce20ce8 P ff6219ada81d4c209dc32acc02b15831: stopping tablet replica
I20260806 21:07:01.671849 4334 raft_consensus.cc:2229] T df52a7d2679d42389ca25f913ce20ce8 P ff6219ada81d4c209dc32acc02b15831 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:07:01.671869 4332 tablet_replica.cc:330] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8: stopping tablet replica
I20260806 21:07:01.671888 4333 tablet_replica.cc:330] T df52a7d2679d42389ca25f913ce20ce8 P 0b706acb694f4024acecc217a6c7a3b5: stopping tablet replica
I20260806 21:07:01.671975 4334 raft_consensus.cc:2258] T df52a7d2679d42389ca25f913ce20ce8 P ff6219ada81d4c209dc32acc02b15831 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:07:01.671988 4332 raft_consensus.cc:2229] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:07:01.672019 4333 raft_consensus.cc:2229] T df52a7d2679d42389ca25f913ce20ce8 P 0b706acb694f4024acecc217a6c7a3b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:07:01.672137 4333 raft_consensus.cc:2258] T df52a7d2679d42389ca25f913ce20ce8 P 0b706acb694f4024acecc217a6c7a3b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:07:01.672216 4332 raft_consensus.cc:2258] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:07:01.672917 4334 ts_tablet_manager.cc:1905] T df52a7d2679d42389ca25f913ce20ce8 P ff6219ada81d4c209dc32acc02b15831: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:07:01.673210 4332 ts_tablet_manager.cc:1905] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:07:01.674567 4333 ts_tablet_manager.cc:1905] T df52a7d2679d42389ca25f913ce20ce8 P 0b706acb694f4024acecc217a6c7a3b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:07:01.676280 4334 ts_tablet_manager.cc:1918] T df52a7d2679d42389ca25f913ce20ce8 P ff6219ada81d4c209dc32acc02b15831: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:07:01.676367 4334 log.cc:1196] T df52a7d2679d42389ca25f913ce20ce8 P ff6219ada81d4c209dc32acc02b15831: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-1/wal/wals/df52a7d2679d42389ca25f913ce20ce8
I20260806 21:07:01.676651 4334 ts_tablet_manager.cc:1939] T df52a7d2679d42389ca25f913ce20ce8 P ff6219ada81d4c209dc32acc02b15831: Deleting consensus metadata
I20260806 21:07:01.677606 4333 ts_tablet_manager.cc:1918] T df52a7d2679d42389ca25f913ce20ce8 P 0b706acb694f4024acecc217a6c7a3b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:07:01.677703 4333 log.cc:1196] T df52a7d2679d42389ca25f913ce20ce8 P 0b706acb694f4024acecc217a6c7a3b5: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/df52a7d2679d42389ca25f913ce20ce8
I20260806 21:07:01.677992 4333 ts_tablet_manager.cc:1939] T df52a7d2679d42389ca25f913ce20ce8 P 0b706acb694f4024acecc217a6c7a3b5: Deleting consensus metadata
I20260806 21:07:01.677778 3767 catalog_manager.cc:4841] TS ff6219ada81d4c209dc32acc02b15831 (127.27.225.194:35821): tablet df52a7d2679d42389ca25f913ce20ce8 (table test-table [id=8fb159bc4645487fa58b6dde884ae762]) successfully deleted
I20260806 21:07:01.678769 3767 catalog_manager.cc:4841] TS 0b706acb694f4024acecc217a6c7a3b5 (127.27.225.195:44843): tablet df52a7d2679d42389ca25f913ce20ce8 (table test-table [id=8fb159bc4645487fa58b6dde884ae762]) successfully deleted
I20260806 21:07:01.680828 4332 ts_tablet_manager.cc:1918] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:07:01.680912 4332 log.cc:1196] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-0/wal/wals/df52a7d2679d42389ca25f913ce20ce8
I20260806 21:07:01.681114 4332 ts_tablet_manager.cc:1939] T df52a7d2679d42389ca25f913ce20ce8 P 871ebb27b0794ec0a464ba6e581a08b8: Deleting consensus metadata
I20260806 21:07:01.681908 3769 catalog_manager.cc:4841] TS 871ebb27b0794ec0a464ba6e581a08b8 (127.27.225.193:32999): tablet df52a7d2679d42389ca25f913ce20ce8 (table test-table [id=8fb159bc4645487fa58b6dde884ae762]) successfully deleted
Aug 06 21:07:01 dist-test-slave-69rg krb5kdc[3735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 21:07:01 dist-test-slave-69rg krb5kdc[3735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:01 dist-test-slave-69rg krb5kdc[3735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 21:07:01.706955 3835 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:01.698971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51860 (local address 127.27.225.254:33337)
0806 21:07:01.699112 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:01.699116 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:01.699383 (+ 267us) server_negotiation.cc:407] Connection header received
0806 21:07:01.699482 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:01.699486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:01.699553 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:01.699685 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:01.700504 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.701504 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.702556 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.702934 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.703071 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.706806 (+ 3735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51860: BlockingRecv error: recv got EOF from 127.0.0.1:51860 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 21:07:01 dist-test-slave-69rg krb5kdc[3735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM
I20260806 21:07:01.720841 3835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.709374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51866 (local address 127.27.225.254:33337)
0806 21:07:01.709519 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:01.709525 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:01.709642 (+ 117us) server_negotiation.cc:407] Connection header received
0806 21:07:01.709772 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:01.709779 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:01.709886 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:01.709980 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:01.710826 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.711793 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.713136 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.713515 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.713619 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.716862 (+ 3243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:01.716867 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:01.716871 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:07:01.716908 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:01.718910 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:01.719460 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:01.719465 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:01.719467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:01.719570 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:01.719877 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:01.719881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:01.719883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:01.720129 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:01.720277 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:01.720526 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 21:07:01.720662 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 21:07:01.723908 3782 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:07:01.724437 3782 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:07:01.729773 4066 tablet_service.cc:1467] Processing CreateTablet for tablet 964c10c0fa5046bbbcdc5732db74ae0d (DEFAULT_TABLE table=test-table [id=51056433cd064c039b0d78e4ebeccb54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:07:01.729773 3924 tablet_service.cc:1467] Processing CreateTablet for tablet 964c10c0fa5046bbbcdc5732db74ae0d (DEFAULT_TABLE table=test-table [id=51056433cd064c039b0d78e4ebeccb54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:07:01.730136 3924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 964c10c0fa5046bbbcdc5732db74ae0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:07:01.730136 4066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 964c10c0fa5046bbbcdc5732db74ae0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:07:01.732686 4310 tablet_bootstrap.cc:492] T 964c10c0fa5046bbbcdc5732db74ae0d P 871ebb27b0794ec0a464ba6e581a08b8: Bootstrap starting.
I20260806 21:07:01.732892 4206 tablet_service.cc:1467] Processing CreateTablet for tablet 964c10c0fa5046bbbcdc5732db74ae0d (DEFAULT_TABLE table=test-table [id=51056433cd064c039b0d78e4ebeccb54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:07:01.733165 4206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 964c10c0fa5046bbbcdc5732db74ae0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:07:01.733587 4310 tablet_bootstrap.cc:654] T 964c10c0fa5046bbbcdc5732db74ae0d P 871ebb27b0794ec0a464ba6e581a08b8: Neither blocks nor log segments found. Creating new log.
I20260806 21:07:01.735405 4310 tablet_bootstrap.cc:492] T 964c10c0fa5046bbbcdc5732db74ae0d P 871ebb27b0794ec0a464ba6e581a08b8: No bootstrap required, opened a new log
I20260806 21:07:01.735510 4310 ts_tablet_manager.cc:1397] T 964c10c0fa5046bbbcdc5732db74ae0d P 871ebb27b0794ec0a464ba6e581a08b8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:07:01.735770 4311 tablet_bootstrap.cc:492] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831: Bootstrap starting.
I20260806 21:07:01.735954 4310 raft_consensus.cc:348] T 964c10c0fa5046bbbcdc5732db74ae0d P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.736135 4310 raft_consensus.cc:374] T 964c10c0fa5046bbbcdc5732db74ae0d P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:07:01.736196 4310 raft_consensus.cc:729] T 964c10c0fa5046bbbcdc5732db74ae0d P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 871ebb27b0794ec0a464ba6e581a08b8, State: Initialized, Role: FOLLOWER
I20260806 21:07:01.736384 4310 consensus_queue.cc:260] T 964c10c0fa5046bbbcdc5732db74ae0d P 871ebb27b0794ec0a464ba6e581a08b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.736699 4310 ts_tablet_manager.cc:1428] T 964c10c0fa5046bbbcdc5732db74ae0d P 871ebb27b0794ec0a464ba6e581a08b8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:07:01.736804 4311 tablet_bootstrap.cc:654] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831: Neither blocks nor log segments found. Creating new log.
I20260806 21:07:01.737382 4309 tablet_bootstrap.cc:492] T 964c10c0fa5046bbbcdc5732db74ae0d P 0b706acb694f4024acecc217a6c7a3b5: Bootstrap starting.
I20260806 21:07:01.738422 4309 tablet_bootstrap.cc:654] T 964c10c0fa5046bbbcdc5732db74ae0d P 0b706acb694f4024acecc217a6c7a3b5: Neither blocks nor log segments found. Creating new log.
I20260806 21:07:01.739800 4309 tablet_bootstrap.cc:492] T 964c10c0fa5046bbbcdc5732db74ae0d P 0b706acb694f4024acecc217a6c7a3b5: No bootstrap required, opened a new log
I20260806 21:07:01.739898 4309 ts_tablet_manager.cc:1397] T 964c10c0fa5046bbbcdc5732db74ae0d P 0b706acb694f4024acecc217a6c7a3b5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:07:01.740382 4309 raft_consensus.cc:348] T 964c10c0fa5046bbbcdc5732db74ae0d P 0b706acb694f4024acecc217a6c7a3b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.740466 4311 tablet_bootstrap.cc:492] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831: No bootstrap required, opened a new log
I20260806 21:07:01.740500 4309 raft_consensus.cc:374] T 964c10c0fa5046bbbcdc5732db74ae0d P 0b706acb694f4024acecc217a6c7a3b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:07:01.740533 4309 raft_consensus.cc:729] T 964c10c0fa5046bbbcdc5732db74ae0d P 0b706acb694f4024acecc217a6c7a3b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b706acb694f4024acecc217a6c7a3b5, State: Initialized, Role: FOLLOWER
I20260806 21:07:01.740556 4311 ts_tablet_manager.cc:1397] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 21:07:01.740654 4309 consensus_queue.cc:260] T 964c10c0fa5046bbbcdc5732db74ae0d P 0b706acb694f4024acecc217a6c7a3b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.740980 4311 raft_consensus.cc:348] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.741109 4311 raft_consensus.cc:374] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:07:01.741145 4311 raft_consensus.cc:729] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff6219ada81d4c209dc32acc02b15831, State: Initialized, Role: FOLLOWER
I20260806 21:07:01.741273 4311 consensus_queue.cc:260] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.741466 4309 ts_tablet_manager.cc:1428] T 964c10c0fa5046bbbcdc5732db74ae0d P 0b706acb694f4024acecc217a6c7a3b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 21:07:01.742420 4311 ts_tablet_manager.cc:1428] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 21:07:01.796171 4350 raft_consensus.cc:482] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:07:01.796446 4350 raft_consensus.cc:504] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.797025 4350 leader_election.cc:288] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 871ebb27b0794ec0a464ba6e581a08b8 (127.27.225.193:32999), 0b706acb694f4024acecc217a6c7a3b5 (127.27.225.195:44843)
I20260806 21:07:01.797474 3944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c10c0fa5046bbbcdc5732db74ae0d" candidate_uuid: "ff6219ada81d4c209dc32acc02b15831" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "871ebb27b0794ec0a464ba6e581a08b8" is_pre_election: true
I20260806 21:07:01.797662 3944 raft_consensus.cc:2454] T 964c10c0fa5046bbbcdc5732db74ae0d P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff6219ada81d4c209dc32acc02b15831 in term 0.
I20260806 21:07:01.797967 4228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c10c0fa5046bbbcdc5732db74ae0d" candidate_uuid: "ff6219ada81d4c209dc32acc02b15831" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b706acb694f4024acecc217a6c7a3b5" is_pre_election: true
I20260806 21:07:01.798053 4019 leader_election.cc:302] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 871ebb27b0794ec0a464ba6e581a08b8, ff6219ada81d4c209dc32acc02b15831; no voters:
I20260806 21:07:01.798170 4228 raft_consensus.cc:2454] T 964c10c0fa5046bbbcdc5732db74ae0d P 0b706acb694f4024acecc217a6c7a3b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff6219ada81d4c209dc32acc02b15831 in term 0.
I20260806 21:07:01.798267 4350 raft_consensus.cc:2781] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:07:01.798357 4350 raft_consensus.cc:482] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:07:01.798411 4350 raft_consensus.cc:3037] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:07:01.799463 4350 raft_consensus.cc:504] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.799772 4350 leader_election.cc:288] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831 [CANDIDATE]: Term 1 election: Requested vote from peers 871ebb27b0794ec0a464ba6e581a08b8 (127.27.225.193:32999), 0b706acb694f4024acecc217a6c7a3b5 (127.27.225.195:44843)
I20260806 21:07:01.800223 3944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c10c0fa5046bbbcdc5732db74ae0d" candidate_uuid: "ff6219ada81d4c209dc32acc02b15831" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "871ebb27b0794ec0a464ba6e581a08b8"
I20260806 21:07:01.800312 4228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "964c10c0fa5046bbbcdc5732db74ae0d" candidate_uuid: "ff6219ada81d4c209dc32acc02b15831" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b706acb694f4024acecc217a6c7a3b5"
I20260806 21:07:01.800401 3944 raft_consensus.cc:3037] T 964c10c0fa5046bbbcdc5732db74ae0d P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:07:01.800472 4228 raft_consensus.cc:3037] T 964c10c0fa5046bbbcdc5732db74ae0d P 0b706acb694f4024acecc217a6c7a3b5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:07:01.801434 3944 raft_consensus.cc:2454] T 964c10c0fa5046bbbcdc5732db74ae0d P 871ebb27b0794ec0a464ba6e581a08b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff6219ada81d4c209dc32acc02b15831 in term 1.
I20260806 21:07:01.801782 4019 leader_election.cc:302] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 871ebb27b0794ec0a464ba6e581a08b8, ff6219ada81d4c209dc32acc02b15831; no voters:
I20260806 21:07:01.801796 4228 raft_consensus.cc:2454] T 964c10c0fa5046bbbcdc5732db74ae0d P 0b706acb694f4024acecc217a6c7a3b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff6219ada81d4c209dc32acc02b15831 in term 1.
I20260806 21:07:01.802050 4350 raft_consensus.cc:2781] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:07:01.802271 4350 raft_consensus.cc:686] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831 [term 1 LEADER]: Becoming Leader. State: Replica: ff6219ada81d4c209dc32acc02b15831, State: Running, Role: LEADER
I20260806 21:07:01.802479 4350 consensus_queue.cc:237] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.804373 3782 catalog_manager.cc:5495] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff6219ada81d4c209dc32acc02b15831 (127.27.225.194). New cstate: current_term: 1 leader_uuid: "ff6219ada81d4c209dc32acc02b15831" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } health_report { overall_health: UNKNOWN } } }
I20260806 21:07:01.821059 4308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:01.817869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57144 (local address 127.27.225.194:35821)
0806 21:07:01.818004 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:01.818007 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:01.818023 (+ 16us) server_negotiation.cc:407] Connection header received
0806 21:07:01.818154 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:01.818157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:01.818203 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:01.818270 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:07:01.818545 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.819134 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.820010 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:01.820257 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:01.820347 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:01.820611 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:07:01.820701 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:07:01.820772 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:01.820891 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 21:07:01.820939 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 21:07:01.823854 4228 raft_consensus.cc:1261] T 964c10c0fa5046bbbcdc5732db74ae0d P 0b706acb694f4024acecc217a6c7a3b5 [term 1 FOLLOWER]: Refusing update from remote peer ff6219ada81d4c209dc32acc02b15831: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:07:01.823872 3944 raft_consensus.cc:1261] T 964c10c0fa5046bbbcdc5732db74ae0d P 871ebb27b0794ec0a464ba6e581a08b8 [term 1 FOLLOWER]: Refusing update from remote peer ff6219ada81d4c209dc32acc02b15831: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:07:01.824381 4350 consensus_queue.cc:1035] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:07:01.824535 4317 consensus_queue.cc:1035] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831 [LEADER]: Connected to new peer: Peer: permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:07:01.834575 3784 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:07:01.834764 3784 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:07:01.836971 3784 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f: Sending DeleteTablet for 3 replicas of tablet 964c10c0fa5046bbbcdc5732db74ae0d
I20260806 21:07:01.837741 3924 tablet_service.cc:1514] Processing DeleteTablet for tablet 964c10c0fa5046bbbcdc5732db74ae0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:07:01 UTC) from {username='oryx', principal='oryx/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:51548
I20260806 21:07:01.837896 4206 tablet_service.cc:1514] Processing DeleteTablet for tablet 964c10c0fa5046bbbcdc5732db74ae0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:07:01 UTC) from {username='oryx', principal='oryx/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:45390
I20260806 21:07:01.837981 4332 tablet_replica.cc:330] T 964c10c0fa5046bbbcdc5732db74ae0d P 871ebb27b0794ec0a464ba6e581a08b8: stopping tablet replica
I20260806 21:07:01.838040 4066 tablet_service.cc:1514] Processing DeleteTablet for tablet 964c10c0fa5046bbbcdc5732db74ae0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:07:01 UTC) from {username='oryx', principal='oryx/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:57138
I20260806 21:07:01.838143 4332 raft_consensus.cc:2229] T 964c10c0fa5046bbbcdc5732db74ae0d P 871ebb27b0794ec0a464ba6e581a08b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:07:01.838232 4334 tablet_replica.cc:330] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831: stopping tablet replica
I20260806 21:07:01.838305 4332 raft_consensus.cc:2258] T 964c10c0fa5046bbbcdc5732db74ae0d P 871ebb27b0794ec0a464ba6e581a08b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:07:01.838347 4334 raft_consensus.cc:2229] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:07:01.838490 4333 tablet_replica.cc:330] T 964c10c0fa5046bbbcdc5732db74ae0d P 0b706acb694f4024acecc217a6c7a3b5: stopping tablet replica
I20260806 21:07:01.838606 4333 raft_consensus.cc:2229] T 964c10c0fa5046bbbcdc5732db74ae0d P 0b706acb694f4024acecc217a6c7a3b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:07:01.838743 4333 raft_consensus.cc:2258] T 964c10c0fa5046bbbcdc5732db74ae0d P 0b706acb694f4024acecc217a6c7a3b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:07:01.838727 4334 raft_consensus.cc:2258] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:07:01.839468 4334 ts_tablet_manager.cc:1905] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:07:01.839485 4332 ts_tablet_manager.cc:1905] T 964c10c0fa5046bbbcdc5732db74ae0d P 871ebb27b0794ec0a464ba6e581a08b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:07:01.839977 3784 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:07:01.840404 3784 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:07:01.842159 4334 ts_tablet_manager.cc:1918] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:07:01.842248 4334 log.cc:1196] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-1/wal/wals/964c10c0fa5046bbbcdc5732db74ae0d
I20260806 21:07:01.842538 4334 ts_tablet_manager.cc:1939] T 964c10c0fa5046bbbcdc5732db74ae0d P ff6219ada81d4c209dc32acc02b15831: Deleting consensus metadata
I20260806 21:07:01.842914 4333 ts_tablet_manager.cc:1905] T 964c10c0fa5046bbbcdc5732db74ae0d P 0b706acb694f4024acecc217a6c7a3b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:07:01.843334 3767 catalog_manager.cc:4841] TS ff6219ada81d4c209dc32acc02b15831 (127.27.225.194:35821): tablet 964c10c0fa5046bbbcdc5732db74ae0d (table test-table [id=51056433cd064c039b0d78e4ebeccb54]) successfully deleted
I20260806 21:07:01.843688 4332 ts_tablet_manager.cc:1918] T 964c10c0fa5046bbbcdc5732db74ae0d P 871ebb27b0794ec0a464ba6e581a08b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:07:01.843789 4332 log.cc:1196] T 964c10c0fa5046bbbcdc5732db74ae0d P 871ebb27b0794ec0a464ba6e581a08b8: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-0/wal/wals/964c10c0fa5046bbbcdc5732db74ae0d
I20260806 21:07:01.844085 4332 ts_tablet_manager.cc:1939] T 964c10c0fa5046bbbcdc5732db74ae0d P 871ebb27b0794ec0a464ba6e581a08b8: Deleting consensus metadata
I20260806 21:07:01.844987 3769 catalog_manager.cc:4841] TS 871ebb27b0794ec0a464ba6e581a08b8 (127.27.225.193:32999): tablet 964c10c0fa5046bbbcdc5732db74ae0d (table test-table [id=51056433cd064c039b0d78e4ebeccb54]) successfully deleted
I20260806 21:07:01.845402 4333 ts_tablet_manager.cc:1918] T 964c10c0fa5046bbbcdc5732db74ae0d P 0b706acb694f4024acecc217a6c7a3b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:07:01.845503 4333 log.cc:1196] T 964c10c0fa5046bbbcdc5732db74ae0d P 0b706acb694f4024acecc217a6c7a3b5: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/964c10c0fa5046bbbcdc5732db74ae0d
I20260806 21:07:01.845733 4333 ts_tablet_manager.cc:1939] T 964c10c0fa5046bbbcdc5732db74ae0d P 0b706acb694f4024acecc217a6c7a3b5: Deleting consensus metadata
I20260806 21:07:01.846685 3767 catalog_manager.cc:4841] TS 0b706acb694f4024acecc217a6c7a3b5 (127.27.225.195:44843): tablet 964c10c0fa5046bbbcdc5732db74ae0d (table test-table [id=51056433cd064c039b0d78e4ebeccb54]) successfully deleted
I20260806 21:07:01.848141 3924 tablet_service.cc:1467] Processing CreateTablet for tablet 3370c5e4b47949bb9fc699c7792318a6 (DEFAULT_TABLE table=test-table [id=fa98496753ef41c8b20f5a1ca1b16041]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:07:01.848469 3924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3370c5e4b47949bb9fc699c7792318a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:07:01.848480 4206 tablet_service.cc:1467] Processing CreateTablet for tablet 3370c5e4b47949bb9fc699c7792318a6 (DEFAULT_TABLE table=test-table [id=fa98496753ef41c8b20f5a1ca1b16041]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:07:01.848753 4206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3370c5e4b47949bb9fc699c7792318a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:07:01.848678 4066 tablet_service.cc:1467] Processing CreateTablet for tablet 3370c5e4b47949bb9fc699c7792318a6 (DEFAULT_TABLE table=test-table [id=fa98496753ef41c8b20f5a1ca1b16041]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:07:01.848968 4066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3370c5e4b47949bb9fc699c7792318a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:07:01.851104 4310 tablet_bootstrap.cc:492] T 3370c5e4b47949bb9fc699c7792318a6 P 871ebb27b0794ec0a464ba6e581a08b8: Bootstrap starting.
I20260806 21:07:01.851228 4309 tablet_bootstrap.cc:492] T 3370c5e4b47949bb9fc699c7792318a6 P 0b706acb694f4024acecc217a6c7a3b5: Bootstrap starting.
I20260806 21:07:01.851735 4311 tablet_bootstrap.cc:492] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831: Bootstrap starting.
I20260806 21:07:01.852025 4310 tablet_bootstrap.cc:654] T 3370c5e4b47949bb9fc699c7792318a6 P 871ebb27b0794ec0a464ba6e581a08b8: Neither blocks nor log segments found. Creating new log.
I20260806 21:07:01.852073 4309 tablet_bootstrap.cc:654] T 3370c5e4b47949bb9fc699c7792318a6 P 0b706acb694f4024acecc217a6c7a3b5: Neither blocks nor log segments found. Creating new log.
I20260806 21:07:01.852742 4311 tablet_bootstrap.cc:654] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831: Neither blocks nor log segments found. Creating new log.
I20260806 21:07:01.853544 4310 tablet_bootstrap.cc:492] T 3370c5e4b47949bb9fc699c7792318a6 P 871ebb27b0794ec0a464ba6e581a08b8: No bootstrap required, opened a new log
I20260806 21:07:01.853632 4310 ts_tablet_manager.cc:1397] T 3370c5e4b47949bb9fc699c7792318a6 P 871ebb27b0794ec0a464ba6e581a08b8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 21:07:01.854112 4310 raft_consensus.cc:348] T 3370c5e4b47949bb9fc699c7792318a6 P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.854254 4310 raft_consensus.cc:374] T 3370c5e4b47949bb9fc699c7792318a6 P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:07:01.854290 4310 raft_consensus.cc:729] T 3370c5e4b47949bb9fc699c7792318a6 P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 871ebb27b0794ec0a464ba6e581a08b8, State: Initialized, Role: FOLLOWER
I20260806 21:07:01.854458 4311 tablet_bootstrap.cc:492] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831: No bootstrap required, opened a new log
I20260806 21:07:01.854425 4310 consensus_queue.cc:260] T 3370c5e4b47949bb9fc699c7792318a6 P 871ebb27b0794ec0a464ba6e581a08b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.854530 4311 ts_tablet_manager.cc:1397] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:07:01.854633 4309 tablet_bootstrap.cc:492] T 3370c5e4b47949bb9fc699c7792318a6 P 0b706acb694f4024acecc217a6c7a3b5: No bootstrap required, opened a new log
I20260806 21:07:01.854696 4310 ts_tablet_manager.cc:1428] T 3370c5e4b47949bb9fc699c7792318a6 P 871ebb27b0794ec0a464ba6e581a08b8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:07:01.854723 4309 ts_tablet_manager.cc:1397] T 3370c5e4b47949bb9fc699c7792318a6 P 0b706acb694f4024acecc217a6c7a3b5: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260806 21:07:01.854925 4311 raft_consensus.cc:348] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.855007 4311 raft_consensus.cc:374] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:07:01.855027 4311 raft_consensus.cc:729] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff6219ada81d4c209dc32acc02b15831, State: Initialized, Role: FOLLOWER
I20260806 21:07:01.855163 4311 consensus_queue.cc:260] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.855226 4309 raft_consensus.cc:348] T 3370c5e4b47949bb9fc699c7792318a6 P 0b706acb694f4024acecc217a6c7a3b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.855345 4309 raft_consensus.cc:374] T 3370c5e4b47949bb9fc699c7792318a6 P 0b706acb694f4024acecc217a6c7a3b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:07:01.855381 4309 raft_consensus.cc:729] T 3370c5e4b47949bb9fc699c7792318a6 P 0b706acb694f4024acecc217a6c7a3b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b706acb694f4024acecc217a6c7a3b5, State: Initialized, Role: FOLLOWER
I20260806 21:07:01.855455 4311 ts_tablet_manager.cc:1428] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:07:01.855515 4309 consensus_queue.cc:260] T 3370c5e4b47949bb9fc699c7792318a6 P 0b706acb694f4024acecc217a6c7a3b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:01.855773 4309 ts_tablet_manager.cc:1428] T 3370c5e4b47949bb9fc699c7792318a6 P 0b706acb694f4024acecc217a6c7a3b5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:07:02.185351 4317 raft_consensus.cc:482] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:07:02.185508 4317 raft_consensus.cc:504] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:02.186049 4317 leader_election.cc:288] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 871ebb27b0794ec0a464ba6e581a08b8 (127.27.225.193:32999), 0b706acb694f4024acecc217a6c7a3b5 (127.27.225.195:44843)
I20260806 21:07:02.186589 3944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3370c5e4b47949bb9fc699c7792318a6" candidate_uuid: "ff6219ada81d4c209dc32acc02b15831" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "871ebb27b0794ec0a464ba6e581a08b8" is_pre_election: true
I20260806 21:07:02.186609 4228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3370c5e4b47949bb9fc699c7792318a6" candidate_uuid: "ff6219ada81d4c209dc32acc02b15831" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b706acb694f4024acecc217a6c7a3b5" is_pre_election: true
I20260806 21:07:02.186769 3944 raft_consensus.cc:2454] T 3370c5e4b47949bb9fc699c7792318a6 P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff6219ada81d4c209dc32acc02b15831 in term 0.
I20260806 21:07:02.186767 4228 raft_consensus.cc:2454] T 3370c5e4b47949bb9fc699c7792318a6 P 0b706acb694f4024acecc217a6c7a3b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff6219ada81d4c209dc32acc02b15831 in term 0.
I20260806 21:07:02.187094 4019 leader_election.cc:302] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 871ebb27b0794ec0a464ba6e581a08b8, ff6219ada81d4c209dc32acc02b15831; no voters:
I20260806 21:07:02.187301 4317 raft_consensus.cc:2781] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:07:02.187377 4317 raft_consensus.cc:482] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:07:02.187413 4317 raft_consensus.cc:3037] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:07:02.188746 4317 raft_consensus.cc:504] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:02.189143 4317 leader_election.cc:288] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831 [CANDIDATE]: Term 1 election: Requested vote from peers 871ebb27b0794ec0a464ba6e581a08b8 (127.27.225.193:32999), 0b706acb694f4024acecc217a6c7a3b5 (127.27.225.195:44843)
I20260806 21:07:02.189654 3944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3370c5e4b47949bb9fc699c7792318a6" candidate_uuid: "ff6219ada81d4c209dc32acc02b15831" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "871ebb27b0794ec0a464ba6e581a08b8"
I20260806 21:07:02.189654 4228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3370c5e4b47949bb9fc699c7792318a6" candidate_uuid: "ff6219ada81d4c209dc32acc02b15831" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b706acb694f4024acecc217a6c7a3b5"
I20260806 21:07:02.189815 3944 raft_consensus.cc:3037] T 3370c5e4b47949bb9fc699c7792318a6 P 871ebb27b0794ec0a464ba6e581a08b8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:07:02.189858 4228 raft_consensus.cc:3037] T 3370c5e4b47949bb9fc699c7792318a6 P 0b706acb694f4024acecc217a6c7a3b5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:07:02.190905 3944 raft_consensus.cc:2454] T 3370c5e4b47949bb9fc699c7792318a6 P 871ebb27b0794ec0a464ba6e581a08b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff6219ada81d4c209dc32acc02b15831 in term 1.
I20260806 21:07:02.191006 4228 raft_consensus.cc:2454] T 3370c5e4b47949bb9fc699c7792318a6 P 0b706acb694f4024acecc217a6c7a3b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff6219ada81d4c209dc32acc02b15831 in term 1.
I20260806 21:07:02.191303 4017 leader_election.cc:302] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b706acb694f4024acecc217a6c7a3b5, ff6219ada81d4c209dc32acc02b15831; no voters:
I20260806 21:07:02.191531 4317 raft_consensus.cc:2781] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:07:02.191612 4317 raft_consensus.cc:686] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831 [term 1 LEADER]: Becoming Leader. State: Replica: ff6219ada81d4c209dc32acc02b15831, State: Running, Role: LEADER
I20260806 21:07:02.191736 4317 consensus_queue.cc:237] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } }
I20260806 21:07:02.193522 3783 catalog_manager.cc:5495] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff6219ada81d4c209dc32acc02b15831 (127.27.225.194). New cstate: current_term: 1 leader_uuid: "ff6219ada81d4c209dc32acc02b15831" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff6219ada81d4c209dc32acc02b15831" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 35821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 } health_report { overall_health: UNKNOWN } } }
I20260806 21:07:02.242501 4308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:02.238982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:35203 (local address 127.27.225.194:35821)
0806 21:07:02.239102 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:02.239106 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:02.239288 (+ 182us) server_negotiation.cc:407] Connection header received
0806 21:07:02.239440 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:02.239443 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:02.239488 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:02.239563 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:07:02.239858 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:02.240361 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:02.241427 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:02.241655 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:02.241795 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:02.242096 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:07:02.242170 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:07:02.242233 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:02.242317 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 21:07:02.242380 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":119,"server-negotiator.queue_time_us":46}
I20260806 21:07:02.242512 4392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:02.238872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:35821 (local address 127.27.225.194:35203)
0806 21:07:02.239205 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:02.239219 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:02.239316 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:02.239618 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:02.239620 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:02.239627 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 21:07:02.239720 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:02.239725 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:02.240590 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:02.240595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:02.241292 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:02.241302 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:02.241793 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:02.241797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:02.241958 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:02.241986 (+ 28us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 21:07:02.242253 (+ 267us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 21:07:02.242258 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 21:07:02.242309 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 21:07:02.242394 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":4,"thread_start_us":79,"threads_started":1}
I20260806 21:07:02.248266 4228 raft_consensus.cc:1261] T 3370c5e4b47949bb9fc699c7792318a6 P 0b706acb694f4024acecc217a6c7a3b5 [term 1 FOLLOWER]: Refusing update from remote peer ff6219ada81d4c209dc32acc02b15831: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:07:02.248307 3944 raft_consensus.cc:1261] T 3370c5e4b47949bb9fc699c7792318a6 P 871ebb27b0794ec0a464ba6e581a08b8 [term 1 FOLLOWER]: Refusing update from remote peer ff6219ada81d4c209dc32acc02b15831: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:07:02.248658 4350 consensus_queue.cc:1035] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b706acb694f4024acecc217a6c7a3b5" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:07:02.248818 4357 consensus_queue.cc:1035] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831 [LEADER]: Connected to new peer: Peer: permanent_uuid: "871ebb27b0794ec0a464ba6e581a08b8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:07:02.282565 3784 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:07:02.282730 3784 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:07:02.284916 3784 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ef1adaf1849477cb101f0574cc34f5f: Sending DeleteTablet for 3 replicas of tablet 3370c5e4b47949bb9fc699c7792318a6
I20260806 21:07:02.285681 3924 tablet_service.cc:1514] Processing DeleteTablet for tablet 3370c5e4b47949bb9fc699c7792318a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:07:02 UTC) from {username='oryx', principal='oryx/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:51548
I20260806 21:07:02.285679 4066 tablet_service.cc:1514] Processing DeleteTablet for tablet 3370c5e4b47949bb9fc699c7792318a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:07:02 UTC) from {username='oryx', principal='oryx/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:57138
I20260806 21:07:02.285728 4206 tablet_service.cc:1514] Processing DeleteTablet for tablet 3370c5e4b47949bb9fc699c7792318a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:07:02 UTC) from {username='oryx', principal='oryx/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:45390
I20260806 21:07:02.285923 4334 tablet_replica.cc:330] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831: stopping tablet replica
I20260806 21:07:02.285923 4332 tablet_replica.cc:330] T 3370c5e4b47949bb9fc699c7792318a6 P 871ebb27b0794ec0a464ba6e581a08b8: stopping tablet replica
I20260806 21:07:02.286054 4334 raft_consensus.cc:2229] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:07:02.286053 4332 raft_consensus.cc:2229] T 3370c5e4b47949bb9fc699c7792318a6 P 871ebb27b0794ec0a464ba6e581a08b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:07:02.286192 4332 raft_consensus.cc:2258] T 3370c5e4b47949bb9fc699c7792318a6 P 871ebb27b0794ec0a464ba6e581a08b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:07:02.286289 4334 raft_consensus.cc:2258] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:07:02.286398 4333 tablet_replica.cc:330] T 3370c5e4b47949bb9fc699c7792318a6 P 0b706acb694f4024acecc217a6c7a3b5: stopping tablet replica
I20260806 21:07:02.286548 4333 raft_consensus.cc:2229] T 3370c5e4b47949bb9fc699c7792318a6 P 0b706acb694f4024acecc217a6c7a3b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:07:02.286707 4333 raft_consensus.cc:2258] T 3370c5e4b47949bb9fc699c7792318a6 P 0b706acb694f4024acecc217a6c7a3b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:07:02.287036 4334 ts_tablet_manager.cc:1905] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:07:02.287338 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 3858
I20260806 21:07:02.287616 4332 ts_tablet_manager.cc:1905] T 3370c5e4b47949bb9fc699c7792318a6 P 871ebb27b0794ec0a464ba6e581a08b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:07:02.289520 4334 ts_tablet_manager.cc:1918] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:07:02.289615 4334 log.cc:1196] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-1/wal/wals/3370c5e4b47949bb9fc699c7792318a6
I20260806 21:07:02.289932 4334 ts_tablet_manager.cc:1939] T 3370c5e4b47949bb9fc699c7792318a6 P ff6219ada81d4c209dc32acc02b15831: Deleting consensus metadata
I20260806 21:07:02.289995 4333 ts_tablet_manager.cc:1905] T 3370c5e4b47949bb9fc699c7792318a6 P 0b706acb694f4024acecc217a6c7a3b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:07:02.290813 3767 catalog_manager.cc:4841] TS ff6219ada81d4c209dc32acc02b15831 (127.27.225.194:35821): tablet 3370c5e4b47949bb9fc699c7792318a6 (table test-table [id=fa98496753ef41c8b20f5a1ca1b16041]) successfully deleted
I20260806 21:07:02.292157 4333 ts_tablet_manager.cc:1918] T 3370c5e4b47949bb9fc699c7792318a6 P 0b706acb694f4024acecc217a6c7a3b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:07:02.292213 4333 log.cc:1196] T 3370c5e4b47949bb9fc699c7792318a6 P 0b706acb694f4024acecc217a6c7a3b5: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/3370c5e4b47949bb9fc699c7792318a6
I20260806 21:07:02.292400 4333 ts_tablet_manager.cc:1939] T 3370c5e4b47949bb9fc699c7792318a6 P 0b706acb694f4024acecc217a6c7a3b5: Deleting consensus metadata
I20260806 21:07:02.293547 3767 catalog_manager.cc:4841] TS 0b706acb694f4024acecc217a6c7a3b5 (127.27.225.195:44843): tablet 3370c5e4b47949bb9fc699c7792318a6 (table test-table [id=fa98496753ef41c8b20f5a1ca1b16041]) successfully deleted
I20260806 21:07:02.297585 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 4000
W20260806 21:07:02.297546 3769 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.225.193:32999 (error 108)
I20260806 21:07:02.299702 4397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:02.299268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:32999 (local address 127.0.0.1:51578)
0806 21:07:02.299531 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:02.299600 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.193:32999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":80,"threads_started":1}
W20260806 21:07:02.299846 3769 catalog_manager.cc:4584] TS 871ebb27b0794ec0a464ba6e581a08b8 (127.27.225.193:32999): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3370c5e4b47949bb9fc699c7792318a6: Network error: Client connection negotiation failed: client connection to 127.27.225.193:32999: connect: Connection refused (error 111)
I20260806 21:07:02.307078 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 4142
I20260806 21:07:02.314855 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 3751
2026-08-06T21:07:02Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3543 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:07:02 dist-test-slave-69rg krb5kdc[4400](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:07:02 dist-test-slave-69rg krb5kdc[4400](info): set up 2 sockets
Aug 06 21:07:02 dist-test-slave-69rg krb5kdc[4400](info): commencing operation
krb5kdc: starting...
W20260806 21:07:03.403132 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:07:03 dist-test-slave-69rg krb5kdc[4400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:07:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:07:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:07:03.537297 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:35209
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:36025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:35209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:07:03.652746 4414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:07:03.653041 4414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:07:03.653086 4414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:07:03.657305 4414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:07:03.657425 4414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:07:03.657490 4414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:07:03.657538 4414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:07:03.657557 4414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:07:03.663213 4414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:36025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:35209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:35209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:07:03.664270 4414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:07:03.665230 4414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:07:03.673980 4422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:03.673987 4420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:03.673995 4419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:03.674521 4414 server_base.cc:1034] running on GCE node
I20260806 21:07:03.675248 4414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:07:03.676471 4414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:07:03.677687 4414 hybrid_clock.cc:648] HybridClock initialized: now 1786050423677660 us; error 49 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T21:07:03Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1368 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:07:03 dist-test-slave-69rg krb5kdc[4427](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:07:03 dist-test-slave-69rg krb5kdc[4427](info): set up 2 sockets
Aug 06 21:07:03 dist-test-slave-69rg krb5kdc[4427](info): commencing operation
krb5kdc: starting...
W20260806 21:07:04.746299 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:07:04 dist-test-slave-69rg krb5kdc[4427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:07:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:07:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:07:04.879040 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:40811
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:40811
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:07:04.992378 4440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:07:04.992720 4440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:07:04.992794 4440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:07:04.996909 4440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:07:04.997002 4440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:07:04.997041 4440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:07:04.997097 4440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:07:04.997138 4440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:07:05.002645 4440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:40811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:40811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:07:05.003819 4440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:07:05.004904 4440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:07:05.014123 4448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:05.014123 4446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:05.014144 4445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:05.014546 4440 server_base.cc:1034] running on GCE node
I20260806 21:07:05.015199 4440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:07:05.016580 4440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:07:05.017774 4440 hybrid_clock.cc:648] HybridClock initialized: now 1786050425017755 us; error 33 us; skew 500 ppm
Aug 06 21:07:05 dist-test-slave-69rg krb5kdc[4427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050425, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:07:05.021054 4440 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:07:05.022169 4440 webserver.cc:460] Webserver started at http://127.27.225.254:33287/ using document root <none> and password file <none>
I20260806 21:07:05.022760 4440 fs_manager.cc:359] Metadata directory not provided
I20260806 21:07:05.022826 4440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:07:05.023005 4440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:07:05.025234 4440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "7cc8cac54321424b824efa880bf0a2a0"
format_stamp: "Formatted at 2026-08-06 21:07:05 on dist-test-slave-69rg"
server_key: "d882fb287284bcb29a4e9c9dbdcf11e8"
server_key_iv: "b31b73deffb1071dabece3065f79519d"
server_key_version: "encryptionkey@0"
I20260806 21:07:05.025913 4440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "7cc8cac54321424b824efa880bf0a2a0"
format_stamp: "Formatted at 2026-08-06 21:07:05 on dist-test-slave-69rg"
server_key: "d882fb287284bcb29a4e9c9dbdcf11e8"
server_key_iv: "b31b73deffb1071dabece3065f79519d"
server_key_version: "encryptionkey@0"
I20260806 21:07:05.030300 4440 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 21:07:05.032809 4455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:05.034252 4440 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:07:05.034406 4440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "7cc8cac54321424b824efa880bf0a2a0"
format_stamp: "Formatted at 2026-08-06 21:07:05 on dist-test-slave-69rg"
server_key: "d882fb287284bcb29a4e9c9dbdcf11e8"
server_key_iv: "b31b73deffb1071dabece3065f79519d"
server_key_version: "encryptionkey@0"
I20260806 21:07:05.034534 4440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:07:05.053393 4440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:07:05.056658 4440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:07:05.056885 4440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:07:05.064527 4440 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:40811
I20260806 21:07:05.064553 4507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:40811 every 8 connection(s)
I20260806 21:07:05.065485 4440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:07:05.066725 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 4440
I20260806 21:07:05.066847 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:07:05.067144 28551 external_mini_cluster.cc:1428] Setting key f2a8d10258ae9698b064b6b797e53bc2
I20260806 21:07:05.068567 4508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:07:05.075479 4508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0: Bootstrap starting.
Aug 06 21:07:05 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:07:05.078001 4508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0: Neither blocks nor log segments found. Creating new log.
I20260806 21:07:05.078984 4508 log.cc:824] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0: Log is configured to *not* fsync() on all Append() calls
I20260806 21:07:05.080920 4508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0: No bootstrap required, opened a new log
I20260806 21:07:05.082252 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:05.068625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48626 (local address 127.27.225.254:40811)
0806 21:07:05.069092 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:05.069112 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:05.069163 (+ 51us) server_negotiation.cc:407] Connection header received
0806 21:07:05.070036 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:05.070058 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:05.070456 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:05.070809 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:05.071498 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:05.073007 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:05.073975 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:05.074690 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:05.074847 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:05.077341 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:05.077347 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:05.077362 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:07:05.077395 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:05.079461 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:05.079927 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:05.079937 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:05.079941 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:05.080047 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:05.080339 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:05.080344 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:05.080345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:05.080733 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:05.080935 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:05.081202 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 21:07:05.081484 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":128,"threads_started":1}
I20260806 21:07:05.085044 4508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc8cac54321424b824efa880bf0a2a0" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40811 } }
I20260806 21:07:05.085300 4508 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:07:05.085381 4508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc8cac54321424b824efa880bf0a2a0, State: Initialized, Role: FOLLOWER
I20260806 21:07:05.085791 4508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc8cac54321424b824efa880bf0a2a0" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40811 } }
I20260806 21:07:05.085999 4508 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:07:05.086086 4508 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:07:05.086184 4508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:07:05.087149 4508 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc8cac54321424b824efa880bf0a2a0" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40811 } }
I20260806 21:07:05.087489 4508 leader_election.cc:302] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cc8cac54321424b824efa880bf0a2a0; no voters:
I20260806 21:07:05.087803 4508 leader_election.cc:288] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:07:05.087932 4513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:07:05.088163 4513 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [term 1 LEADER]: Becoming Leader. State: Replica: 7cc8cac54321424b824efa880bf0a2a0, State: Running, Role: LEADER
I20260806 21:07:05.088533 4513 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc8cac54321424b824efa880bf0a2a0" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40811 } }
I20260806 21:07:05.089097 4508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:07:05.090552 4514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7cc8cac54321424b824efa880bf0a2a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc8cac54321424b824efa880bf0a2a0" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40811 } } }
I20260806 21:07:05.090690 4514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [sys.catalog]: This master's current role is: LEADER
I20260806 21:07:05.091351 4515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cc8cac54321424b824efa880bf0a2a0. Latest consensus state: current_term: 1 leader_uuid: "7cc8cac54321424b824efa880bf0a2a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc8cac54321424b824efa880bf0a2a0" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40811 } } }
I20260806 21:07:05.091583 4515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [sys.catalog]: This master's current role is: LEADER
W20260806 21:07:05.094062 4529 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:07:05.094130 4529 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:07:05.094220 4523 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:07:05.095002 4523 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:07:05.100447 4523 catalog_manager.cc:1347] Generated new cluster ID: baac17a5a0084c8fb18be94a88db2690
I20260806 21:07:05.100533 4523 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:07:05.117468 4523 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:07:05.118824 4523 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:07:05.139740 4523 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0: Generated new TSK 0
I20260806 21:07:05.140614 4523 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.225.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
I20260806 21:07:05.174829 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40811
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:07:05.290719 4534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:07:05.291095 4534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:07:05.291177 4534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:07:05.295459 4534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:07:05.295573 4534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:07:05.295676 4534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:07:05.301362 4534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:07:05.302832 4534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:07:05.303961 4534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:07:05.314198 4539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:05.314116 4540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:05.314172 4534 server_base.cc:1034] running on GCE node
W20260806 21:07:05.314157 4542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:05.314891 4534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:07:05.315689 4534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:07:05.316876 4534 hybrid_clock.cc:648] HybridClock initialized: now 1786050425316847 us; error 43 us; skew 500 ppm
Aug 06 21:07:05 dist-test-slave-69rg krb5kdc[4427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050425, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:07:05.325387 4534 init.cc:377] Logged in from keytab as kudu/127.27.225.193@KRBTEST.COM (short username kudu)
I20260806 21:07:05.326515 4534 webserver.cc:460] Webserver started at http://127.27.225.193:39791/ using document root <none> and password file <none>
I20260806 21:07:05.327124 4534 fs_manager.cc:359] Metadata directory not provided
I20260806 21:07:05.327227 4534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:07:05.327507 4534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:07:05.329653 4534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "022270d18a684b25b7e677d83ffca989"
format_stamp: "Formatted at 2026-08-06 21:07:05 on dist-test-slave-69rg"
server_key: "61fe391cfd109b4a8922e687dc151fd9"
server_key_iv: "ff63ab21ea198c000bd878f65f6a8118"
server_key_version: "encryptionkey@0"
I20260806 21:07:05.330336 4534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "022270d18a684b25b7e677d83ffca989"
format_stamp: "Formatted at 2026-08-06 21:07:05 on dist-test-slave-69rg"
server_key: "61fe391cfd109b4a8922e687dc151fd9"
server_key_iv: "ff63ab21ea198c000bd878f65f6a8118"
server_key_version: "encryptionkey@0"
I20260806 21:07:05.334493 4534 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 21:07:05.337272 4549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:05.338689 4534 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:07:05.338833 4534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "022270d18a684b25b7e677d83ffca989"
format_stamp: "Formatted at 2026-08-06 21:07:05 on dist-test-slave-69rg"
server_key: "61fe391cfd109b4a8922e687dc151fd9"
server_key_iv: "ff63ab21ea198c000bd878f65f6a8118"
server_key_version: "encryptionkey@0"
I20260806 21:07:05.338991 4534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:07:05.356509 4534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:07:05.360117 4534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:07:05.360373 4534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:07:05.361047 4534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:07:05.362054 4534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:07:05.362133 4534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:05.362221 4534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:07:05.362278 4534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:05.372289 4534 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:34861
I20260806 21:07:05.372309 4665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:34861 every 8 connection(s)
I20260806 21:07:05.373294 4534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
Aug 06 21:07:05 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050425, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:07:05.383301 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 4534
I20260806 21:07:05.383472 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:07:05.383826 28551 external_mini_cluster.cc:1428] Setting key 4bd41336d73ab160a308ccadf63f35f3
I20260806 21:07:05.388517 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:05.375263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:53463 (local address 127.27.225.254:40811)
0806 21:07:05.375431 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:05.375436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:05.376244 (+ 808us) server_negotiation.cc:407] Connection header received
0806 21:07:05.377676 (+ 1432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:05.377679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:05.377741 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:05.377843 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:05.379105 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:05.379663 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:05.380821 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:05.381072 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:05.381191 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:05.384174 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:05.384177 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:05.384179 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:07:05.384205 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:05.385717 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:05.386449 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:05.386452 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:05.386453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:05.386532 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:05.387063 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:05.387066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:05.387068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:05.387284 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:05.387376 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:05.388158 (+ 782us) server_negotiation.cc:299] Negotiation successful
0806 21:07:05.388315 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 21:07:05.389454 4668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:05.375493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.193:53463)
0806 21:07:05.376029 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:05.376074 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:05.377435 (+ 1361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:05.378016 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:05.378023 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:05.378434 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:05.378937 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:05.378949 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:05.379801 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:05.379804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:05.380686 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:05.380696 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:05.381240 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:05.381244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:05.381456 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:05.382129 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:05.382148 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:05.383959 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:07:05.385879 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:05.385888 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:05.385905 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:05.386327 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:05.386629 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:05.386636 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:05.386641 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:05.386873 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:05.387406 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:07:05.387417 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:07:05.387821 (+ 404us) client_negotiation.cc:769] Sending connection context
0806 21:07:05.388094 (+ 273us) client_negotiation.cc:240] Negotiation successful
0806 21:07:05.388483 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":127,"threads_started":1}
I20260806 21:07:05.391072 4666 heartbeater.cc:347] Connected to a master server at 127.27.225.254:40811
I20260806 21:07:05.391382 4666 heartbeater.cc:464] Registering TS with master...
I20260806 21:07:05.392081 4666 heartbeater.cc:511] Master 127.27.225.254:40811 requested a full tablet report, sending...
I20260806 21:07:05.394071 4472 ts_manager.cc:194] Registered new tserver with Master: 022270d18a684b25b7e677d83ffca989 (127.27.225.193:34861)
I20260806 21:07:05.395320 4472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.193@KRBTEST.COM'} at 127.27.225.193:53463
WARNING: no policy specified for kudu/127.27.225.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
I20260806 21:07:05.417761 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40811
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:07:05.532806 4671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:07:05.533146 4671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:07:05.533226 4671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:07:05.537484 4671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:07:05.537612 4671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:07:05.537709 4671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:07:05.543941 4671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:07:05.545169 4671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:07:05.546304 4671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:07:05.557041 4677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:05.557076 4680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:05.557096 4676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:05.557341 4671 server_base.cc:1034] running on GCE node
I20260806 21:07:05.557801 4671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:07:05.558631 4671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:07:05.559845 4671 hybrid_clock.cc:648] HybridClock initialized: now 1786050425559807 us; error 51 us; skew 500 ppm
Aug 06 21:07:05 dist-test-slave-69rg krb5kdc[4427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050425, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:07:05.563349 4671 init.cc:377] Logged in from keytab as kudu/127.27.225.194@KRBTEST.COM (short username kudu)
I20260806 21:07:05.564345 4671 webserver.cc:460] Webserver started at http://127.27.225.194:44831/ using document root <none> and password file <none>
I20260806 21:07:05.564989 4671 fs_manager.cc:359] Metadata directory not provided
I20260806 21:07:05.565055 4671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:07:05.565414 4671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:07:05.567742 4671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "75abfb1040c34dbd97153a481674d4e9"
format_stamp: "Formatted at 2026-08-06 21:07:05 on dist-test-slave-69rg"
server_key: "672a36a7ae5ee76297de0fab8db0f970"
server_key_iv: "94f8c3a4df0a47ff8a2a57a50ae9b46b"
server_key_version: "encryptionkey@0"
I20260806 21:07:05.568321 4671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "75abfb1040c34dbd97153a481674d4e9"
format_stamp: "Formatted at 2026-08-06 21:07:05 on dist-test-slave-69rg"
server_key: "672a36a7ae5ee76297de0fab8db0f970"
server_key_iv: "94f8c3a4df0a47ff8a2a57a50ae9b46b"
server_key_version: "encryptionkey@0"
I20260806 21:07:05.572458 4671 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 21:07:05.575251 4687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:05.576655 4671 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 21:07:05.576824 4671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "75abfb1040c34dbd97153a481674d4e9"
format_stamp: "Formatted at 2026-08-06 21:07:05 on dist-test-slave-69rg"
server_key: "672a36a7ae5ee76297de0fab8db0f970"
server_key_iv: "94f8c3a4df0a47ff8a2a57a50ae9b46b"
server_key_version: "encryptionkey@0"
I20260806 21:07:05.576959 4671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:07:05.592697 4671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:07:05.595844 4671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:07:05.596058 4671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:07:05.596688 4671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:07:05.597572 4671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:07:05.597651 4671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:05.597726 4671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:07:05.597785 4671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:05.607281 4671 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:42179
I20260806 21:07:05.607302 4802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:42179 every 8 connection(s)
I20260806 21:07:05.608322 4671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:07:05.615957 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 4671
I20260806 21:07:05.616108 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:07:05.616470 28551 external_mini_cluster.cc:1428] Setting key 4d001c8d8474cd48bdf42581a79ad35a
Aug 06 21:07:05 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050425, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:07:05.624629 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:05.610325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:59987 (local address 127.27.225.254:40811)
0806 21:07:05.610484 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:05.610488 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:05.611227 (+ 739us) server_negotiation.cc:407] Connection header received
0806 21:07:05.612782 (+ 1555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:05.612785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:05.612852 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:05.612941 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:05.614285 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:05.614879 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:05.616442 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:05.616754 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:05.616843 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:05.619994 (+ 3151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:05.619998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:05.620000 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:07:05.620026 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:05.621960 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:05.622756 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:05.622759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:05.622761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:05.622846 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:05.623294 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:05.623297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:05.623298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:05.623503 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:05.623583 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:05.624382 (+ 799us) server_negotiation.cc:299] Negotiation successful
0806 21:07:05.624501 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 21:07:05.626200 4805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:05.610594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.194:59987)
0806 21:07:05.611049 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:05.611096 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:05.612513 (+ 1417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:05.613133 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:05.613141 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:05.613535 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:05.614123 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:05.614138 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:05.615027 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:05.615036 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:05.616282 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:05.616296 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:05.616936 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:05.616940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:05.617145 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:05.617914 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:05.617954 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:05.619787 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:07:05.622118 (+ 2331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:05.622129 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:05.622152 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:05.622624 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:05.622946 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:05.622953 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:05.622956 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:05.623175 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:05.623617 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:07:05.623627 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:07:05.624052 (+ 425us) client_negotiation.cc:769] Sending connection context
0806 21:07:05.624323 (+ 271us) client_negotiation.cc:240] Negotiation successful
0806 21:07:05.624777 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":109,"threads_started":1}
I20260806 21:07:05.627799 4803 heartbeater.cc:347] Connected to a master server at 127.27.225.254:40811
I20260806 21:07:05.628116 4803 heartbeater.cc:464] Registering TS with master...
I20260806 21:07:05.628865 4803 heartbeater.cc:511] Master 127.27.225.254:40811 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.27.225.195@KRBTEST.COM; defaulting to no policy
I20260806 21:07:05.630297 4472 ts_manager.cc:194] Registered new tserver with Master: 75abfb1040c34dbd97153a481674d4e9 (127.27.225.194:42179)
I20260806 21:07:05.631127 4472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.194@KRBTEST.COM'} at 127.27.225.194:59987
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
I20260806 21:07:05.652959 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40811
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:07:05.769829 4808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:07:05.770107 4808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:07:05.770156 4808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:07:05.774282 4808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:07:05.774360 4808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:07:05.774423 4808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:07:05.779974 4808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:07:05.781211 4808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:07:05.782366 4808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:07:05.792725 4816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:05.792725 4814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:05.792732 4813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:05.793051 4808 server_base.cc:1034] running on GCE node
I20260806 21:07:05.793501 4808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:07:05.794317 4808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:07:05.795517 4808 hybrid_clock.cc:648] HybridClock initialized: now 1786050425795500 us; error 39 us; skew 500 ppm
Aug 06 21:07:05 dist-test-slave-69rg krb5kdc[4427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050425, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:07:05.798619 4808 init.cc:377] Logged in from keytab as kudu/127.27.225.195@KRBTEST.COM (short username kudu)
I20260806 21:07:05.799611 4808 webserver.cc:460] Webserver started at http://127.27.225.195:41469/ using document root <none> and password file <none>
I20260806 21:07:05.800237 4808 fs_manager.cc:359] Metadata directory not provided
I20260806 21:07:05.800343 4808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:07:05.800585 4808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:07:05.802824 4808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "a28c3cb25fd84207bb7fac1633bd19a9"
format_stamp: "Formatted at 2026-08-06 21:07:05 on dist-test-slave-69rg"
server_key: "ded4fe507a5fe0e1ad6fe63cdfc90468"
server_key_iv: "876adf9fcb99244696686a735832a465"
server_key_version: "encryptionkey@0"
I20260806 21:07:05.803411 4808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "a28c3cb25fd84207bb7fac1633bd19a9"
format_stamp: "Formatted at 2026-08-06 21:07:05 on dist-test-slave-69rg"
server_key: "ded4fe507a5fe0e1ad6fe63cdfc90468"
server_key_iv: "876adf9fcb99244696686a735832a465"
server_key_version: "encryptionkey@0"
I20260806 21:07:05.807359 4808 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:07:05.809971 4823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:05.811352 4808 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:07:05.811528 4808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "a28c3cb25fd84207bb7fac1633bd19a9"
format_stamp: "Formatted at 2026-08-06 21:07:05 on dist-test-slave-69rg"
server_key: "ded4fe507a5fe0e1ad6fe63cdfc90468"
server_key_iv: "876adf9fcb99244696686a735832a465"
server_key_version: "encryptionkey@0"
I20260806 21:07:05.811676 4808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:07:05.823889 4808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:07:05.826866 4808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:07:05.827066 4808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:07:05.827672 4808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:07:05.828493 4808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:07:05.828547 4808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:05.828615 4808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:07:05.828660 4808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:05.838199 4808 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:34613
I20260806 21:07:05.838289 4939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:34613 every 8 connection(s)
I20260806 21:07:05.839298 4808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:07:05.841121 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 4808
I20260806 21:07:05.841336 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:07:05.841672 28551 external_mini_cluster.cc:1428] Setting key f4fed47a5075cacb8745cc16f5e32e42
Aug 06 21:07:05 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050425, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:07:05.854282 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:05.841461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:36537 (local address 127.27.225.254:40811)
0806 21:07:05.841606 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:05.841609 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:05.842668 (+ 1059us) server_negotiation.cc:407] Connection header received
0806 21:07:05.844381 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:05.844383 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:05.844431 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:05.844500 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:05.845733 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:05.846273 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:05.847418 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:05.847640 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:05.847738 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:05.850439 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:05.850442 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:05.850444 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:07:05.850470 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:05.851908 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:05.852667 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:05.852669 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:05.852671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:05.852763 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:05.853129 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:05.853131 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:05.853132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:05.853352 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:05.853439 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:05.854016 (+ 577us) server_negotiation.cc:299] Negotiation successful
0806 21:07:05.854128 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 21:07:05.855329 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:05.842014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.195:36537)
0806 21:07:05.842516 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:05.842568 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:05.844184 (+ 1616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:05.844665 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:05.844673 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:05.845025 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:05.845571 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:05.845586 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:05.846393 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:05.846395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:05.847290 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:05.847300 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:05.847820 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:05.847823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:05.847973 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:05.848577 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:05.848593 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:05.850262 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:07:05.852049 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:05.852055 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:05.852072 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:05.852555 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:05.852874 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:05.852876 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:05.852877 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:05.853039 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:05.853450 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:07:05.853455 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:07:05.853716 (+ 261us) client_negotiation.cc:769] Sending connection context
0806 21:07:05.853976 (+ 260us) client_negotiation.cc:240] Negotiation successful
0806 21:07:05.854300 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":140,"threads_started":1}
I20260806 21:07:05.856580 4940 heartbeater.cc:347] Connected to a master server at 127.27.225.254:40811
I20260806 21:07:05.856841 4940 heartbeater.cc:464] Registering TS with master...
I20260806 21:07:05.857437 4940 heartbeater.cc:511] Master 127.27.225.254:40811 requested a full tablet report, sending...
I20260806 21:07:05.858562 4471 ts_manager.cc:194] Registered new tserver with Master: a28c3cb25fd84207bb7fac1633bd19a9 (127.27.225.195:34613)
I20260806 21:07:05.859261 4471 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.195@KRBTEST.COM'} at 127.27.225.195:36537
I20260806 21:07:05.868554 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.27.225.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.225.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.225.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.193.keytab.
Entry for principal oryx/127.27.225.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.193.keytab.
WARNING: no policy specified for oryx/127.27.225.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.225.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.225.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.194.keytab.
Entry for principal oryx/127.27.225.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.194.keytab.
WARNING: no policy specified for oryx/127.27.225.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.225.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.225.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.195.keytab.
Entry for principal oryx/127.27.225.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.195.keytab.
I20260806 21:07:05.953595 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 4534
I20260806 21:07:05.960069 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 4671
I20260806 21:07:05.965942 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 4808
I20260806 21:07:05.971691 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 4440
I20260806 21:07:05.979153 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:40811
--webserver_interface=127.27.225.254
--webserver_port=33287
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:40811
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:07:06.106060 4949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:07:06.106386 4949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:07:06.106431 4949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:07:06.110749 4949 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:07:06.110832 4949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:07:06.110852 4949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:07:06.110870 4949 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:07:06.110885 4949 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:07:06.116737 4949 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:40811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:40811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=33287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:07:06.117990 4949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:07:06.119122 4949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:07:06.127992 4955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:06.127992 4958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:06.128191 4949 server_base.cc:1034] running on GCE node
W20260806 21:07:06.128006 4956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:06.128870 4949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:07:06.130041 4949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:07:06.131228 4949 hybrid_clock.cc:648] HybridClock initialized: now 1786050426131196 us; error 50 us; skew 500 ppm
Aug 06 21:07:06 dist-test-slave-69rg krb5kdc[4427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050426, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:07:06.134119 4949 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:07:06.135058 4949 webserver.cc:460] Webserver started at http://127.27.225.254:33287/ using document root <none> and password file <none>
I20260806 21:07:06.135628 4949 fs_manager.cc:359] Metadata directory not provided
I20260806 21:07:06.135754 4949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:07:06.139772 4949 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 21:07:06.142103 4965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:06.143445 4949 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 21:07:06.143697 4949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "7cc8cac54321424b824efa880bf0a2a0"
format_stamp: "Formatted at 2026-08-06 21:07:05 on dist-test-slave-69rg"
server_key: "d882fb287284bcb29a4e9c9dbdcf11e8"
server_key_iv: "b31b73deffb1071dabece3065f79519d"
server_key_version: "encryptionkey@0"
I20260806 21:07:06.144304 4949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:07:06.165138 4949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:07:06.168470 4949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:07:06.168701 4949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:07:06.176676 4949 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:40811
I20260806 21:07:06.176693 5018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:40811 every 8 connection(s)
I20260806 21:07:06.177840 4949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:07:06.182278 5019 sys_catalog.cc:263] Verifying existing consensus state
I20260806 21:07:06.184003 5019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0: Bootstrap starting.
I20260806 21:07:06.187083 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 4949
I20260806 21:07:06.187546 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:34861
--local_ip_for_outbound_sockets=127.27.225.193
--tserver_master_addrs=127.27.225.254:40811
--webserver_port=39791
--webserver_interface=127.27.225.193
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.27.225.193
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:07:06.201720 5019 log.cc:824] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0: Log is configured to *not* fsync() on all Append() calls
I20260806 21:07:06.208698 5019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 21:07:06.209133 5019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0: Bootstrap complete.
I20260806 21:07:06.213498 5019 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc8cac54321424b824efa880bf0a2a0" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40811 } }
I20260806 21:07:06.214035 5019 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc8cac54321424b824efa880bf0a2a0, State: Initialized, Role: FOLLOWER
I20260806 21:07:06.214560 5019 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc8cac54321424b824efa880bf0a2a0" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40811 } }
I20260806 21:07:06.214730 5019 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:07:06.214828 5019 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:07:06.214951 5019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [term 1 FOLLOWER]: Advancing to term 2
I20260806 21:07:06.217150 5019 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc8cac54321424b824efa880bf0a2a0" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40811 } }
I20260806 21:07:06.217558 5019 leader_election.cc:302] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cc8cac54321424b824efa880bf0a2a0; no voters:
I20260806 21:07:06.217993 5019 leader_election.cc:288] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 21:07:06.218253 5023 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 21:07:06.218487 5023 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [term 2 LEADER]: Becoming Leader. State: Replica: 7cc8cac54321424b824efa880bf0a2a0, State: Running, Role: LEADER
I20260806 21:07:06.219074 5023 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc8cac54321424b824efa880bf0a2a0" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40811 } }
I20260806 21:07:06.219847 5019 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:07:06.220944 5025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cc8cac54321424b824efa880bf0a2a0. Latest consensus state: current_term: 2 leader_uuid: "7cc8cac54321424b824efa880bf0a2a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc8cac54321424b824efa880bf0a2a0" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40811 } } }
I20260806 21:07:06.221104 5025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [sys.catalog]: This master's current role is: LEADER
I20260806 21:07:06.221555 5032 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:07:06.221560 5024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7cc8cac54321424b824efa880bf0a2a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc8cac54321424b824efa880bf0a2a0" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40811 } } }
I20260806 21:07:06.221709 5024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0 [sys.catalog]: This master's current role is: LEADER
I20260806 21:07:06.225492 5032 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:07:06.226348 5032 catalog_manager.cc:1259] Loaded cluster ID: baac17a5a0084c8fb18be94a88db2690
I20260806 21:07:06.226438 5032 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:07:06.228319 5032 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:07:06.228991 5032 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7cc8cac54321424b824efa880bf0a2a0: Loaded TSK: 0
I20260806 21:07:06.230135 5032 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 21:07:06.319818 5021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:07:06.320369 5021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:07:06.320501 5021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:07:06.326179 5021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:07:06.326313 5021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:07:06.326414 5021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:07:06.332160 5021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.193.keytab
--principal=oryx/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:34861
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=39791
--tserver_master_addrs=127.27.225.254:40811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:07:06.333335 5021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:07:06.334656 5021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:07:06.345741 5046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:06.345834 5048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:06.345778 5045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:06.346424 5021 server_base.cc:1034] running on GCE node
I20260806 21:07:06.346972 5021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:07:06.347896 5021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:07:06.349293 5021 hybrid_clock.cc:648] HybridClock initialized: now 1786050426349263 us; error 55 us; skew 500 ppm
Aug 06 21:07:06 dist-test-slave-69rg krb5kdc[4427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050426, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:07:06.352267 5021 init.cc:377] Logged in from keytab as oryx/127.27.225.193@KRBTEST.COM (short username oryx)
I20260806 21:07:06.353262 5021 webserver.cc:460] Webserver started at http://127.27.225.193:39791/ using document root <none> and password file <none>
I20260806 21:07:06.353935 5021 fs_manager.cc:359] Metadata directory not provided
I20260806 21:07:06.354022 5021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:07:06.358367 5021 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:07:06.360636 5055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:06.362222 5021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:07:06.362402 5021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "022270d18a684b25b7e677d83ffca989"
format_stamp: "Formatted at 2026-08-06 21:07:05 on dist-test-slave-69rg"
server_key: "61fe391cfd109b4a8922e687dc151fd9"
server_key_iv: "ff63ab21ea198c000bd878f65f6a8118"
server_key_version: "encryptionkey@0"
I20260806 21:07:06.363096 5021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:07:06.400300 5021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:07:06.404685 5021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:07:06.404937 5021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:07:06.405644 5021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:07:06.406697 5021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:07:06.406780 5021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:06.406862 5021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:07:06.406895 5021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:06.417330 5021 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:34861
I20260806 21:07:06.417351 5171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:34861 every 8 connection(s)
I20260806 21:07:06.418637 5021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:07:06.426436 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 5021
I20260806 21:07:06.426935 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:42179
--local_ip_for_outbound_sockets=127.27.225.194
--tserver_master_addrs=127.27.225.254:40811
--webserver_port=44831
--webserver_interface=127.27.225.194
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.27.225.194
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 21:07:06 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:06 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:06.434001 5175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:06.421345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.193:52749)
0806 21:07:06.422012 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:06.422059 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:06.423524 (+ 1465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:06.424780 (+ 1256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:06.424788 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:06.425196 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:06.425705 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:06.425728 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.427319 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.427322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:06.428248 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:06.428260 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.429467 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.429471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:06.429623 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:06.430363 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:06.430381 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:06.433112 (+ 2731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":150,"threads_started":1}
W20260806 21:07:06.434501 5172 heartbeater.cc:644] Failed to heartbeat to 127.27.225.254:40811 (0 consecutive failures): Not authorized: Failed to ping master at 127.27.225.254:40811: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
W20260806 21:07:06.434494 5174 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:06.421170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:52749 (local address 127.27.225.254:40811)
0806 21:07:06.421697 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:06.421721 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:06.422203 (+ 482us) server_negotiation.cc:407] Connection header received
0806 21:07:06.423979 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:06.423991 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:06.424331 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:06.424640 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:06.425987 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.427137 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.428399 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.429249 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.429476 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:06.433478 (+ 4002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.193:52749: BlockingRecv error: recv got EOF from 127.27.225.193:52749 (error 108)
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":108,"threads_started":1}
Aug 06 21:07:06 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:06 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:06.440413 5175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:06.435219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.193:45593)
0806 21:07:06.435321 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:06.435336 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:06.435418 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:06.435650 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:06.435653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:06.435817 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:06.435910 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:06.435915 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.436655 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.436660 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:06.437200 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:06.437208 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.437679 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.437682 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:06.437784 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:06.438362 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:06.438373 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:06.440273 (+ 1900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 21:07:06.440479 5174 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:06.435314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:45593 (local address 127.27.225.254:40811)
0806 21:07:06.435427 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:06.435431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:06.435447 (+ 16us) server_negotiation.cc:407] Connection header received
0806 21:07:06.435493 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:06.435496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:06.435547 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:06.435635 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:06.436024 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.436546 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.437330 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.437562 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.437669 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:06.440374 (+ 2705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.193:45593: BlockingRecv error: recv got EOF from 127.27.225.193:45593 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 06 21:07:06 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:06 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:06.446163 5175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:06.440952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.193:34279)
0806 21:07:06.441040 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:06.441054 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:06.441126 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:06.441344 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:06.441347 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:06.441494 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:06.441563 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:06.441568 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.442373 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.442379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:06.442915 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:06.442923 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.443388 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.443391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:06.443478 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:06.443948 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:06.443959 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:06.446011 (+ 2052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 21:07:06.446375 5172 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 21:07:06.446265 5174 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:06.441032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:34279 (local address 127.27.225.254:40811)
0806 21:07:06.441135 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:06.441139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:06.441153 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:07:06.441195 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:06.441198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:06.441244 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:06.441331 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:06.441674 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.442266 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.443040 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.443270 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.443373 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:06.446136 (+ 2763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.193:34279: BlockingRecv error: recv got EOF from 127.27.225.193:34279 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260806 21:07:06.546938 5176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:07:06.547353 5176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:07:06.547438 5176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:07:06.551914 5176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:07:06.552018 5176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:07:06.552117 5176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:07:06.557969 5176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.194.keytab
--principal=oryx/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:42179
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=44831
--tserver_master_addrs=127.27.225.254:40811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:07:06.559171 5176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:07:06.560262 5176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:07:06.571192 5181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:06.571231 5182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:06.571261 5184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:06.571410 5176 server_base.cc:1034] running on GCE node
I20260806 21:07:06.571842 5176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:07:06.572549 5176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:07:06.573758 5176 hybrid_clock.cc:648] HybridClock initialized: now 1786050426573734 us; error 30 us; skew 500 ppm
Aug 06 21:07:06 dist-test-slave-69rg krb5kdc[4427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050426, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:07:06.576869 5176 init.cc:377] Logged in from keytab as oryx/127.27.225.194@KRBTEST.COM (short username oryx)
I20260806 21:07:06.577793 5176 webserver.cc:460] Webserver started at http://127.27.225.194:44831/ using document root <none> and password file <none>
I20260806 21:07:06.578436 5176 fs_manager.cc:359] Metadata directory not provided
I20260806 21:07:06.578516 5176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:07:06.583091 5176 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 21:07:06.585453 5191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:06.586941 5176 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:07:06.587106 5176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "75abfb1040c34dbd97153a481674d4e9"
format_stamp: "Formatted at 2026-08-06 21:07:05 on dist-test-slave-69rg"
server_key: "672a36a7ae5ee76297de0fab8db0f970"
server_key_iv: "94f8c3a4df0a47ff8a2a57a50ae9b46b"
server_key_version: "encryptionkey@0"
I20260806 21:07:06.587734 5176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:07:06.603645 5176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:07:06.607455 5176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:07:06.607693 5176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:07:06.608357 5176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:07:06.609251 5176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:07:06.609324 5176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:06.609400 5176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:07:06.609450 5176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:06.619428 5176 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:42179
I20260806 21:07:06.619450 5306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:42179 every 8 connection(s)
I20260806 21:07:06.620668 5176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:07:06.625918 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 5176
I20260806 21:07:06.626410 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:34613
--local_ip_for_outbound_sockets=127.27.225.195
--tserver_master_addrs=127.27.225.254:40811
--webserver_port=41469
--webserver_interface=127.27.225.195
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.27.225.195
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 21:07:06 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:06 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 21:07:06.633081 5174 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:06.622894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:34219 (local address 127.27.225.254:40811)
0806 21:07:06.623044 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:06.623048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:06.623933 (+ 885us) server_negotiation.cc:407] Connection header received
0806 21:07:06.625363 (+ 1430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:06.625366 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:06.625436 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:06.625571 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:06.627007 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.627586 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.628787 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.629011 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.629120 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:06.632895 (+ 3775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.194:34219: BlockingRecv error: recv got EOF from 127.27.225.194:34219 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 21:07:06.633577 5309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:06.623264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.194:34219)
0806 21:07:06.623761 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:06.623808 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:06.625131 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:06.625750 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:06.625757 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:06.626256 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:06.626844 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:06.626859 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.627723 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.627728 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:06.628639 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:06.628649 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.629179 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.629183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:06.629354 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:06.630042 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:06.630058 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:06.632664 (+ 2606us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":124,"threads_started":1}
W20260806 21:07:06.634258 5307 heartbeater.cc:644] Failed to heartbeat to 127.27.225.254:40811 (0 consecutive failures): Not authorized: Failed to ping master at 127.27.225.254:40811: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Aug 06 21:07:06 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:06 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:06.641175 5309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:06.634790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.194:36337)
0806 21:07:06.634905 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:06.634921 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:06.635006 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:06.635333 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:06.635335 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:06.635483 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:06.635583 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:06.635588 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.636757 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.636760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:06.637335 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:06.637344 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.638051 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.638054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:06.638152 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:06.638805 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:06.638818 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:06.640954 (+ 2136us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 21:07:06.641295 5174 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:06.634872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:36337 (local address 127.27.225.254:40811)
0806 21:07:06.635027 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:06.635034 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:06.635054 (+ 20us) server_negotiation.cc:407] Connection header received
0806 21:07:06.635110 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:06.635116 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:06.635176 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:06.635293 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:06.635702 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.636615 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.637463 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.637899 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.638040 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:06.641155 (+ 3115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.194:36337: BlockingRecv error: recv got EOF from 127.27.225.194:36337 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 21:07:06 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:06 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:06.647943 5309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:06.641797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.194:42983)
0806 21:07:06.641971 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:06.641989 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:06.642085 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:06.642371 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:06.642374 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:06.642516 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:06.642602 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:06.642607 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.643691 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.643694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:06.644297 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:06.644307 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.644948 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.644951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:06.645018 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:06.645624 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:06.645636 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:06.647832 (+ 2196us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":89}
W20260806 21:07:06.648267 5307 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 21:07:06.648327 5174 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:06.641938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:42983 (local address 127.27.225.254:40811)
0806 21:07:06.642128 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:06.642133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:06.642148 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:07:06.642204 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:06.642206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:06.642252 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:06.642337 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:06.642721 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.643548 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.644439 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.644808 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.644915 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:06.648028 (+ 3113us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.194:42983: BlockingRecv error: recv got EOF from 127.27.225.194:42983 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":107,"threads_started":1}
W20260806 21:07:06.751323 5310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:07:06.751636 5310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:07:06.751681 5310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:07:06.755990 5310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:07:06.756084 5310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:07:06.756150 5310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:07:06.761783 5310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/krb5kdc/oryx_127.27.225.195.keytab
--principal=oryx/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:34613
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=41469
--tserver_master_addrs=127.27.225.254:40811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:07:06.762987 5310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:07:06.764884 5310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:07:06.776327 5320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:06.776448 5317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:06.776327 5318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:06.776434 5310 server_base.cc:1034] running on GCE node
I20260806 21:07:06.777271 5310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:07:06.778096 5310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:07:06.779302 5310 hybrid_clock.cc:648] HybridClock initialized: now 1786050426779282 us; error 35 us; skew 500 ppm
Aug 06 21:07:06 dist-test-slave-69rg krb5kdc[4427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050426, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:07:06.783293 5310 init.cc:377] Logged in from keytab as oryx/127.27.225.195@KRBTEST.COM (short username oryx)
I20260806 21:07:06.784498 5310 webserver.cc:460] Webserver started at http://127.27.225.195:41469/ using document root <none> and password file <none>
I20260806 21:07:06.785069 5310 fs_manager.cc:359] Metadata directory not provided
I20260806 21:07:06.785152 5310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:07:06.789264 5310 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:07:06.791550 5327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:06.793054 5310 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 21:07:06.793206 5310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "a28c3cb25fd84207bb7fac1633bd19a9"
format_stamp: "Formatted at 2026-08-06 21:07:05 on dist-test-slave-69rg"
server_key: "ded4fe507a5fe0e1ad6fe63cdfc90468"
server_key_iv: "876adf9fcb99244696686a735832a465"
server_key_version: "encryptionkey@0"
I20260806 21:07:06.793783 5310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:07:06.808845 5310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:07:06.812875 5310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:07:06.813231 5310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:07:06.813982 5310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:07:06.814895 5310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:07:06.814944 5310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:06.815008 5310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:07:06.815042 5310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:06.825402 5310 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:34613
I20260806 21:07:06.825436 5441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:34613 every 8 connection(s)
I20260806 21:07:06.826501 5310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:07:06.835075 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 5310
Aug 06 21:07:06 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:06 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 21:07:06.838798 5174 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:06.828568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:40959 (local address 127.27.225.254:40811)
0806 21:07:06.828733 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:06.828737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:06.829632 (+ 895us) server_negotiation.cc:407] Connection header received
0806 21:07:06.831093 (+ 1461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:06.831096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:06.831174 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:06.831282 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:06.832478 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.833007 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.834201 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.834429 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.834550 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:06.838657 (+ 4107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.195:40959: BlockingRecv error: recv got EOF from 127.27.225.195:40959 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 21:07:06.839427 5444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:06.828976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.195:40959)
0806 21:07:06.829469 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:06.829512 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:06.830870 (+ 1358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:06.831468 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:06.831475 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:06.831853 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:06.832323 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:06.832336 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.833137 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.833140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:06.834067 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:06.834076 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.834590 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.834595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:06.834815 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:06.835558 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:06.835585 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:06.838460 (+ 2875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":128,"threads_started":1}
W20260806 21:07:06.840056 5442 heartbeater.cc:644] Failed to heartbeat to 127.27.225.254:40811 (0 consecutive failures): Not authorized: Failed to ping master at 127.27.225.254:40811: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Aug 06 21:07:06 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:06 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:06.846303 5444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:06.840559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.195:41039)
0806 21:07:06.840679 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:06.840695 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:06.840774 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:06.840970 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:06.840972 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:06.841114 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:06.841203 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:06.841208 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.842034 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.842037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:06.842758 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:06.842768 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.843389 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.843392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:06.843476 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:06.843962 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:06.843974 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:06.846136 (+ 2162us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 21:07:06.846393 5174 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:06.840651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:41039 (local address 127.27.225.254:40811)
0806 21:07:06.840780 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:06.840784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:06.840805 (+ 21us) server_negotiation.cc:407] Connection header received
0806 21:07:06.840836 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:06.840838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:06.840875 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:06.840942 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:06.841357 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.841915 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.842904 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.843247 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.843368 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:06.846271 (+ 2903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.195:41039: BlockingRecv error: recv got EOF from 127.27.225.195:41039 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 21:07:06.846647 5311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:06.836280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48638 (local address 127.27.225.254:40811)
0806 21:07:06.836433 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:06.836449 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:06.836463 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:07:06.836507 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:06.836509 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:06.836547 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:06.836659 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:06.837635 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.838530 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.839627 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.839954 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.840072 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:06.841941 (+ 1869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:06.841951 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:06.841975 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:07:06.842016 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:06.843942 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:06.844493 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:06.844498 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:06.844504 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:06.844675 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:06.845107 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:06.845112 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:06.845114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:06.845556 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:06.845809 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:06.846180 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 21:07:06.846486 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 21:07:06 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:06 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:06.852706 5444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:06.847023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.195:44423)
0806 21:07:06.847164 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:06.847176 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:06.847244 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:06.847545 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:06.847547 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:06.847734 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:06.847832 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:06.847840 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.848559 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.848564 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:06.849307 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:06.849316 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.849854 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.849860 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:06.849957 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:06.850470 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:06.850483 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:06.852575 (+ 2092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 21:07:06.852887 5311 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:06.847196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:44423 (local address 127.27.225.254:40811)
0806 21:07:06.847325 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:06.847330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:06.847342 (+ 12us) server_negotiation.cc:407] Connection header received
0806 21:07:06.847375 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:06.847378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:06.847416 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:06.847497 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:06.847943 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.848460 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.849430 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:06.849686 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:06.849794 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:06.852726 (+ 2932us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.195:44423: BlockingRecv error: recv got EOF from 127.27.225.195:44423 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260806 21:07:06.853025 5442 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 21:07:07 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:07 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:07.453372 5446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:07.447104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.193:52655)
0806 21:07:07.447420 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:07.447440 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:07.447588 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:07.447883 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:07.447886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:07.448118 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:07.448243 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:07.448249 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:07.449148 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:07.449151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:07.450044 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:07.450056 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:07.450667 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:07.450670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:07.450782 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:07.451369 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:07.451384 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:07.453250 (+ 1866us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":84,"threads_started":1}
W20260806 21:07:07.453531 5447 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:07.447273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:52655 (local address 127.27.225.254:40811)
0806 21:07:07.447607 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:07.447611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:07.447626 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:07:07.447673 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:07.447675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:07.447716 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:07.447798 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:07.448401 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:07.449022 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:07.450194 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:07.450540 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:07.450649 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:07.453392 (+ 2743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.193:52655: BlockingRecv error: recv got EOF from 127.27.225.193:52655 (error 108)
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":85,"threads_started":1}
Aug 06 21:07:07 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:07 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:07.654903 5448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:07.648934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.194:35355)
0806 21:07:07.649230 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:07.649252 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:07.649353 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:07.649647 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:07.649650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:07.649880 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:07.649982 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:07.649988 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:07.650790 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:07.650793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:07.651404 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:07.651414 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:07.652032 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:07.652039 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:07.652139 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:07.652759 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:07.652770 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:07.654772 (+ 2002us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":106,"threads_started":1}
W20260806 21:07:07.655138 5447 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:07.649030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:35355 (local address 127.27.225.254:40811)
0806 21:07:07.649230 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:07.649235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:07.649307 (+ 72us) server_negotiation.cc:407] Connection header received
0806 21:07:07.649446 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:07.649448 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:07.649493 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:07.649557 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:07.650099 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:07.650655 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:07.651528 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:07.651884 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:07.651971 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:07.654998 (+ 3027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.194:35355: BlockingRecv error: recv got EOF from 127.27.225.194:35355 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 21:07:07 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:07 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:07.860222 5449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:07.853897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.195:34265)
0806 21:07:07.854319 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:07.854332 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:07.854476 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:07.854809 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:07.854812 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:07.855039 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:07.855166 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:07.855172 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:07.855974 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:07.855980 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:07.856574 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:07.856583 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:07.857097 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:07.857101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:07.857209 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:07.857902 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:07.857926 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:07.860096 (+ 2170us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":93,"threads_started":1}
W20260806 21:07:07.860415 5447 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:07.854064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:34265 (local address 127.27.225.254:40811)
0806 21:07:07.854255 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:07.854261 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:07.854420 (+ 159us) server_negotiation.cc:407] Connection header received
0806 21:07:07.854596 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:07.854599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:07.854684 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:07.854784 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:07.855304 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:07.855856 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:07.856727 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:07.856962 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:07.857088 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:07.860246 (+ 3158us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.195:34265: BlockingRecv error: recv got EOF from 127.27.225.195:34265 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 06 21:07:08 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:08 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:08.460481 5450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:08.454301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.193:35147)
0806 21:07:08.454592 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:08.454609 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:08.454716 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:08.455024 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:08.455026 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:08.455225 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:08.455334 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:08.455341 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:08.456231 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:08.456234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:08.456831 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:08.456841 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:08.457428 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:08.457432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:08.457508 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:08.458096 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:08.458108 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:08.460344 (+ 2236us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":105,"threads_started":1}
W20260806 21:07:08.460733 5451 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:08.454401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:35147 (local address 127.27.225.254:40811)
0806 21:07:08.454696 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:08.454700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:08.454712 (+ 12us) server_negotiation.cc:407] Connection header received
0806 21:07:08.454813 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:08.454815 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:08.454871 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:08.454969 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:08.455482 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:08.456039 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:08.456984 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:08.457261 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:08.457366 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:08.460522 (+ 3156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.193:35147: BlockingRecv error: recv got EOF from 127.27.225.193:35147 (error 108)
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":99,"threads_started":1}
Aug 06 21:07:08 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:08 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:08.662457 5452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:08.655887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.194:39925)
0806 21:07:08.656305 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:08.656325 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:08.656502 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:08.656848 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:08.656851 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:08.657085 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:08.657194 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:08.657200 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:08.658178 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:08.658185 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:08.658819 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:08.658828 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:08.659380 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:08.659384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:08.659484 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:08.660147 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:08.660160 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:08.662241 (+ 2081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":126,"threads_started":1}
W20260806 21:07:08.662521 5451 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:08.656065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:39925 (local address 127.27.225.254:40811)
0806 21:07:08.656289 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:08.656295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:08.656428 (+ 133us) server_negotiation.cc:407] Connection header received
0806 21:07:08.656653 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:08.656657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:08.656718 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:08.656824 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:08.657337 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:08.658036 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:08.658969 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:08.659237 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:08.659362 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:08.662395 (+ 3033us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.194:39925: BlockingRecv error: recv got EOF from 127.27.225.194:39925 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 06 21:07:08 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:08 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:08.867476 5453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:08.861146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.195:54055)
0806 21:07:08.861527 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:08.861541 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:08.861635 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:08.861897 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:08.861899 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:08.862084 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:08.862179 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:08.862184 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:08.863131 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:08.863135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:08.863712 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:08.863722 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:08.864328 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:08.864331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:08.864408 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:08.865051 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:08.865069 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:08.867284 (+ 2215us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":94,"threads_started":1}
W20260806 21:07:08.867550 5451 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:08.861519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:54055 (local address 127.27.225.254:40811)
0806 21:07:08.861674 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:08.861678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:08.861691 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:07:08.861733 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:08.861735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:08.861778 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:08.861877 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:08.862334 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:08.862944 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:08.863867 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:08.864173 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:08.864280 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:08.867433 (+ 3153us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.195:54055: BlockingRecv error: recv got EOF from 127.27.225.195:54055 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 21:07:09 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:09 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:09.468355 5454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:09.461506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.193:35539)
0806 21:07:09.461803 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:09.461841 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:09.461977 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:09.462277 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:09.462279 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:09.462487 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:09.462594 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:09.462601 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:09.463558 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:09.463562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:09.464164 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:09.464175 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:09.464784 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:09.464789 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:09.464861 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:09.465635 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:09.465652 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:09.468173 (+ 2521us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":115,"threads_started":1}
W20260806 21:07:09.468484 5455 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:09.461695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:35539 (local address 127.27.225.254:40811)
0806 21:07:09.461966 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:09.461970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:09.461986 (+ 16us) server_negotiation.cc:407] Connection header received
0806 21:07:09.462054 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:09.462056 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:09.462099 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:09.462193 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:09.462785 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:09.463384 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:09.464333 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:09.464612 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:09.464723 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:09.468343 (+ 3620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.193:35539: BlockingRecv error: recv got EOF from 127.27.225.193:35539 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":142,"threads_started":1}
Aug 06 21:07:09 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:09 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:09.669447 5456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:09.663319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.194:49459)
0806 21:07:09.663609 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:09.663631 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:09.663733 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:09.663974 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:09.663976 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:09.664201 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:09.664307 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:09.664313 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:09.665097 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:09.665100 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:09.665696 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:09.665706 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:09.666389 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:09.666393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:09.666466 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:09.667290 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:09.667312 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:09.669317 (+ 2005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":109,"threads_started":1}
W20260806 21:07:09.669782 5455 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:09.663467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:49459 (local address 127.27.225.254:40811)
0806 21:07:09.663618 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:09.663622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:09.663705 (+ 83us) server_negotiation.cc:407] Connection header received
0806 21:07:09.663815 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:09.663818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:09.663861 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:09.663937 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:09.664421 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:09.664974 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:09.665905 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:09.666252 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:09.666352 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:09.669637 (+ 3285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.194:49459: BlockingRecv error: recv got EOF from 127.27.225.194:49459 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 21:07:09 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:09 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:09.874083 5457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:09.868330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.195:48657)
0806 21:07:09.868641 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:09.868656 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:09.868804 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:09.869071 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:09.869074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:09.869307 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:09.869411 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:09.869417 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:09.870187 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:09.870190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:09.870835 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:09.870845 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:09.871309 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:09.871312 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:09.871392 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:09.871905 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:09.871918 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:09.873953 (+ 2035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":126,"threads_started":1}
W20260806 21:07:09.874323 5455 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:09.868413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:48657 (local address 127.27.225.254:40811)
0806 21:07:09.868548 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:09.868552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:09.868786 (+ 234us) server_negotiation.cc:407] Connection header received
0806 21:07:09.868903 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:09.868905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:09.868956 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:09.869029 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:09.869528 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:09.870078 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:09.870958 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:09.871174 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:09.871262 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:09.874155 (+ 2893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.195:48657: BlockingRecv error: recv got EOF from 127.27.225.195:48657 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 21:07:10 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:10 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:10.479840 5458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:10.469347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.193:59919)
0806 21:07:10.469722 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:10.469742 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:10.469901 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:10.470274 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:10.470277 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:10.470488 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:10.470628 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:10.470636 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:10.471702 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:10.471705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:10.472583 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:10.472597 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:10.473061 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:10.473063 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:10.473124 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:10.477638 (+ 4514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:10.477651 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:10.479709 (+ 2058us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":89,"threads_started":1}
W20260806 21:07:10.480010 5459 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:10.469601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:59919 (local address 127.27.225.254:40811)
0806 21:07:10.469991 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:10.469995 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:10.470010 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:07:10.470054 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:10.470056 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:10.470106 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:10.470201 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:10.470781 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:10.471520 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:10.472732 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:10.472957 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:10.473043 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:10.479870 (+ 6827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.193:59919: BlockingRecv error: recv got EOF from 127.27.225.193:59919 (error 108)
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":121,"threads_started":1}
Aug 06 21:07:10 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:10 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:10.676441 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:10.670423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.194:47399)
0806 21:07:10.670730 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:10.670744 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:10.670840 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:10.671125 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:10.671128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:10.671371 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:10.671476 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:10.671482 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:10.672304 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:10.672308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:10.672877 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:10.672885 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:10.673341 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:10.673344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:10.673409 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:10.674000 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:10.674011 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:10.676274 (+ 2263us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":74,"threads_started":1}
W20260806 21:07:10.676785 5459 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:10.670590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:47399 (local address 127.27.225.254:40811)
0806 21:07:10.670820 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:10.670824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:10.670840 (+ 16us) server_negotiation.cc:407] Connection header received
0806 21:07:10.670920 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:10.670922 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:10.670966 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:10.671041 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:10.671610 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:10.672169 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:10.672990 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:10.673218 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:10.673312 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:10.676474 (+ 3162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.194:47399: BlockingRecv error: recv got EOF from 127.27.225.194:47399 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Aug 06 21:07:10 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:10 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:10.881006 5461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:10.875195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.195:33105)
0806 21:07:10.875473 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:10.875486 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:10.875588 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:10.875942 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:10.875944 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:10.876127 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:10.876220 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:10.876225 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:10.877155 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:10.877166 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:10.877729 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:10.877738 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:10.878277 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:10.878281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:10.878424 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:10.878941 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:10.878953 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:10.880877 (+ 1924us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":79,"threads_started":1}
W20260806 21:07:10.881160 5459 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:10.875292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:33105 (local address 127.27.225.254:40811)
0806 21:07:10.875487 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:10.875495 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:10.875560 (+ 65us) server_negotiation.cc:407] Connection header received
0806 21:07:10.875723 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:10.875730 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:10.875800 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:10.875879 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:10.876350 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:10.877005 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:10.877904 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:10.878143 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:10.878266 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:10.881033 (+ 2767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.195:33105: BlockingRecv error: recv got EOF from 127.27.225.195:33105 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 06 21:07:11 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:11 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:11.487217 5462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:11.480865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.193:45843)
0806 21:07:11.481201 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:11.481220 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:11.481329 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:11.481667 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:11.481669 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:11.481896 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:11.481997 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:11.482003 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:11.482813 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:11.482817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:11.483471 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:11.483480 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:11.483970 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:11.483974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:11.484065 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:11.484650 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:11.484662 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:11.487053 (+ 2391us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":86,"threads_started":1}
W20260806 21:07:11.487370 5463 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:11.480925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:45843 (local address 127.27.225.254:40811)
0806 21:07:11.481279 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:11.481284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:11.481303 (+ 19us) server_negotiation.cc:407] Connection header received
0806 21:07:11.481452 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:11.481456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:11.481515 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:11.481622 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:11.482134 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:11.482682 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:11.483592 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:11.483833 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:11.483915 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:11.487203 (+ 3288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.193:45843: BlockingRecv error: recv got EOF from 127.27.225.193:45843 (error 108)
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":69,"threads_started":1}
Aug 06 21:07:11 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:11 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:11.682979 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:11.677325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.194:34387)
0806 21:07:11.677694 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:11.677713 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:11.677819 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:11.678156 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:11.678158 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:11.678354 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:11.678446 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:11.678451 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:11.679276 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:11.679279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:11.679841 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:11.679849 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:11.680288 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:11.680293 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:11.680390 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:11.680977 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:11.680988 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:11.682863 (+ 1875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":131,"threads_started":1}
W20260806 21:07:11.683174 5463 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:11.677544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:34387 (local address 127.27.225.254:40811)
0806 21:07:11.677764 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:11.677768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:11.677807 (+ 39us) server_negotiation.cc:407] Connection header received
0806 21:07:11.677943 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:11.677946 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:11.677991 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:11.678078 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:11.678565 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:11.679167 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:11.679943 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:11.680155 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:11.680239 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:11.683004 (+ 2765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.194:34387: BlockingRecv error: recv got EOF from 127.27.225.194:34387 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 06 21:07:11 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:11 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:11.888019 5465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:11.882018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.195:60457)
0806 21:07:11.882364 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:11.882383 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:11.882497 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:11.882790 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:11.882793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:11.883025 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:11.883147 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:11.883154 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:11.883943 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:11.883947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:11.884534 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:11.884542 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:11.885008 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:11.885012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:11.885102 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:11.885696 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:11.885709 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:11.887895 (+ 2186us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":131,"threads_started":1}
W20260806 21:07:11.888257 5463 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:11.882232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:60457 (local address 127.27.225.254:40811)
0806 21:07:11.882423 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:11.882430 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:11.882447 (+ 17us) server_negotiation.cc:407] Connection header received
0806 21:07:11.882614 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:11.882617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:11.882675 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:11.882767 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:11.883279 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:11.883825 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:11.884657 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:11.884887 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:11.885003 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:11.888079 (+ 3076us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.195:60457: BlockingRecv error: recv got EOF from 127.27.225.195:60457 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 06 21:07:12 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:12 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:12.494707 5466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:12.488063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.193:38353)
0806 21:07:12.488371 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:12.488386 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:12.488482 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:12.488888 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:12.488890 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:12.489094 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:12.489197 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:12.489202 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:12.490112 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:12.490115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:12.491022 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:12.491033 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:12.491892 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:12.491902 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:12.492030 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:12.492551 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:12.492563 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:12.494553 (+ 1990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":123,"threads_started":1}
W20260806 21:07:12.494877 5467 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:12.488307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:38353 (local address 127.27.225.254:40811)
0806 21:07:12.488588 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:12.488593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:12.488608 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:07:12.488669 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:12.488671 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:12.488717 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:12.488816 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:12.489348 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:12.489992 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:12.491171 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:12.491512 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:12.491660 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:12.494717 (+ 3057us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.193:38353: BlockingRecv error: recv got EOF from 127.27.225.193:38353 (error 108)
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":110,"threads_started":1}
Aug 06 21:07:12 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:12 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:12.690407 5468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:12.683916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.194:48497)
0806 21:07:12.684220 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:12.684244 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:12.684354 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:12.684699 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:12.684701 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:12.684970 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:12.685100 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:12.685108 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:12.686111 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:12.686114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:12.686929 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:12.686940 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:12.687433 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:12.687435 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:12.687501 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:12.688068 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:12.688081 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:12.690278 (+ 2197us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":123,"threads_started":1}
W20260806 21:07:12.690711 5467 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:12.684051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:48497 (local address 127.27.225.254:40811)
0806 21:07:12.684207 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:12.684211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:12.684326 (+ 115us) server_negotiation.cc:407] Connection header received
0806 21:07:12.684499 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:12.684502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:12.684555 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:12.684654 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:12.685289 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:12.685994 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:12.687048 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:12.687317 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:12.687410 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:12.690586 (+ 3176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.194:48497: BlockingRecv error: recv got EOF from 127.27.225.194:48497 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 21:07:12 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:12 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:12.895061 5469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:12.888936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.195:48575)
0806 21:07:12.889267 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:12.889284 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:12.889392 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:12.889645 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:12.889648 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:12.889898 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:12.890000 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:12.890006 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:12.890849 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:12.890852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:12.891535 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:12.891569 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:12.892088 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:12.892092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:12.892172 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:12.892711 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:12.892724 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:12.894933 (+ 2209us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":119,"threads_started":1}
W20260806 21:07:12.895300 5467 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:12.889035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:48575 (local address 127.27.225.254:40811)
0806 21:07:12.889191 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:12.889198 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:12.889414 (+ 216us) server_negotiation.cc:407] Connection header received
0806 21:07:12.889472 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:12.889474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:12.889529 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:12.889605 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:12.890174 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:12.890720 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:12.891705 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:12.891948 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:12.892088 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:12.895172 (+ 3084us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.195:48575: BlockingRecv error: recv got EOF from 127.27.225.195:48575 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 21:07:13 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:13 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:13.502864 5470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:13.495747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.193:46845)
0806 21:07:13.496117 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:13.496132 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:13.496236 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:13.496524 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:13.496526 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:13.496728 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:13.496828 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:13.496834 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:13.497799 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:13.497803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:13.498657 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:13.498672 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:13.499258 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:13.499262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:13.499362 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:13.500025 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:13.500040 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:13.502683 (+ 2643us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":115,"threads_started":1}
W20260806 21:07:13.502974 5471 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:13.495854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:46845 (local address 127.27.225.254:40811)
0806 21:07:13.496203 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:13.496208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:13.496226 (+ 18us) server_negotiation.cc:407] Connection header received
0806 21:07:13.496332 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:13.496335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:13.496384 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:13.496481 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:13.496996 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:13.497647 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:13.498817 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:13.499061 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:13.499194 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:13.502832 (+ 3638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.193:46845: BlockingRecv error: recv got EOF from 127.27.225.193:46845 (error 108)
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":125,"threads_started":1}
Aug 06 21:07:13 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:13 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:13.698130 5473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:13.691818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.194:58087)
0806 21:07:13.692185 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:13.692199 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:13.692379 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:13.692741 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:13.692746 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:13.693000 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:13.693100 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:13.693106 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:13.694020 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:13.694025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:13.694600 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:13.694609 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:13.695117 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:13.695120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:13.695207 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:13.695780 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:13.695792 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:13.697992 (+ 2200us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":141,"threads_started":1}
W20260806 21:07:13.698488 5471 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:13.691886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:58087 (local address 127.27.225.254:40811)
0806 21:07:13.692073 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:13.692078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:13.692314 (+ 236us) server_negotiation.cc:407] Connection header received
0806 21:07:13.692516 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:13.692518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:13.692577 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:13.692671 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:13.693258 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:13.693900 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:13.694727 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:13.694989 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:13.695112 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:13.698185 (+ 3073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.194:58087: BlockingRecv error: recv got EOF from 127.27.225.194:58087 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 06 21:07:13 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:13 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:13.902074 5474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:13.895949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.195:42559)
0806 21:07:13.896303 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:13.896318 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:13.896419 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:13.896667 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:13.896669 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:13.896859 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:13.896956 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:13.896961 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:13.897772 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:13.897775 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:13.898637 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:13.898670 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:13.899268 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:13.899272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:13.899369 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:13.900018 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:13.900030 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:13.901959 (+ 1929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":120,"threads_started":1}
W20260806 21:07:13.902240 5471 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:13.896095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:42559 (local address 127.27.225.254:40811)
0806 21:07:13.896247 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:13.896257 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:13.896390 (+ 133us) server_negotiation.cc:407] Connection header received
0806 21:07:13.896502 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:13.896504 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:13.896554 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:13.896634 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:13.897067 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:13.897644 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:13.898781 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:13.899103 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:13.899214 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:13.902110 (+ 2896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.195:42559: BlockingRecv error: recv got EOF from 127.27.225.195:42559 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 21:07:14 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:14 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:14.509308 5475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:14.503846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.193:60871)
0806 21:07:14.504134 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:14.504148 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:14.504256 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:14.504502 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:14.504504 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:14.504690 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:14.504809 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:14.504815 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:14.505540 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:14.505542 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:14.506134 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:14.506144 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:14.506576 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:14.506579 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:14.506642 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:14.507163 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:14.507175 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:14.509141 (+ 1966us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":118,"threads_started":1}
W20260806 21:07:14.509529 5476 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:14.504004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:60871 (local address 127.27.225.254:40811)
0806 21:07:14.504282 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:14.504285 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:14.504315 (+ 30us) server_negotiation.cc:407] Connection header received
0806 21:07:14.504349 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:14.504351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:14.504393 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:14.504463 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:14.504918 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:14.505440 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:14.506247 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:14.506472 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:14.506554 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:14.509339 (+ 2785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.193:60871: BlockingRecv error: recv got EOF from 127.27.225.193:60871 (error 108)
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":126,"threads_started":1}
Aug 06 21:07:14 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:14 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:14.705533 5477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:14.699023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.194:42629)
0806 21:07:14.699440 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:14.699454 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:14.699562 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:14.700020 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:14.700024 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:14.700271 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:14.700392 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:14.700398 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:14.701438 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:14.701441 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:14.702138 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:14.702149 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:14.702660 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:14.702663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:14.702756 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:14.703351 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:14.703363 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:14.705404 (+ 2041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":124,"threads_started":1}
W20260806 21:07:14.705904 5476 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:14.699188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:42629 (local address 127.27.225.254:40811)
0806 21:07:14.699386 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:14.699391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:14.699539 (+ 148us) server_negotiation.cc:407] Connection header received
0806 21:07:14.699722 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:14.699725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:14.699825 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:14.699939 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:14.700546 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:14.701219 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:14.702288 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:14.702530 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:14.702638 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:14.705739 (+ 3101us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.194:42629: BlockingRecv error: recv got EOF from 127.27.225.194:42629 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 21:07:14 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:14 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:14.908887 5478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:14.903104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.195:50809)
0806 21:07:14.903418 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:14.903433 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:14.903543 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:14.903830 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:14.903833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:14.904024 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:14.904132 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:14.904138 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:14.904956 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:14.904959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:14.905570 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:14.905579 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:14.906141 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:14.906144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:14.906212 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:14.906741 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:14.906753 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:14.908708 (+ 1955us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":124,"threads_started":1}
W20260806 21:07:14.908989 5476 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:14.903182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:50809 (local address 127.27.225.254:40811)
0806 21:07:14.903348 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:14.903353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:14.903541 (+ 188us) server_negotiation.cc:407] Connection header received
0806 21:07:14.903621 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:14.903624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:14.903676 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:14.903755 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:14.904284 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:14.904836 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:14.905710 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:14.905956 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:14.906048 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:14.908860 (+ 2812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.195:50809: BlockingRecv error: recv got EOF from 127.27.225.195:50809 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 21:07:15 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:15 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:15.520998 5479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:15.510316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.193:60947)
0806 21:07:15.510771 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:15.510785 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:15.510885 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:15.511367 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:15.511370 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:15.511634 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:15.511773 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:15.511781 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:15.512641 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:15.512646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:15.513216 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:15.513226 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:15.513725 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:15.513729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:15.513851 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:15.518663 (+ 4812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:15.518678 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:15.520793 (+ 2115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":107,"threads_started":1}
W20260806 21:07:15.521159 5480 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:15.510435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:60947 (local address 127.27.225.254:40811)
0806 21:07:15.510717 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:15.510723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:15.510864 (+ 141us) server_negotiation.cc:407] Connection header received
0806 21:07:15.511118 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:15.511125 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:15.511188 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:15.511341 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:15.511906 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:15.512524 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:15.513347 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:15.513593 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:15.513703 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:15.520996 (+ 7293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.193:60947: BlockingRecv error: recv got EOF from 127.27.225.193:60947 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":84,"threads_started":1}
Aug 06 21:07:15 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:15 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:15.712985 5481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:15.706367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.194:49179)
0806 21:07:15.706768 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:15.706783 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:15.706925 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:15.707240 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:15.707243 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:15.707478 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:15.707585 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:15.707591 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:15.708420 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:15.708425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:15.709020 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:15.709029 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:15.709705 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:15.709708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:15.709803 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:15.710474 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:15.710486 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:15.712830 (+ 2344us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":99,"threads_started":1}
W20260806 21:07:15.713173 5480 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:15.706490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:49179 (local address 127.27.225.254:40811)
0806 21:07:15.706701 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:15.706708 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:15.706867 (+ 159us) server_negotiation.cc:407] Connection header received
0806 21:07:15.707048 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:15.707051 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:15.707112 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:15.707208 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:15.707712 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:15.708297 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:15.709312 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:15.709564 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:15.709682 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:15.713031 (+ 3349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.194:49179: BlockingRecv error: recv got EOF from 127.27.225.194:49179 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 06 21:07:15 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:15 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:15.918385 5482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:15.910072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.195:40037)
0806 21:07:15.910457 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:15.910483 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:15.910586 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:15.910981 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:15.910984 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:15.911253 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:15.911355 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:15.911361 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:15.912664 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:15.912668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:15.913877 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:15.913891 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:15.914548 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:15.914552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:15.914832 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:15.915705 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:15.915718 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:15.918253 (+ 2535us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":84,"threads_started":1}
W20260806 21:07:15.918664 5480 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:15.910326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:40037 (local address 127.27.225.254:40811)
0806 21:07:15.910563 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:15.910567 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:15.910586 (+ 19us) server_negotiation.cc:407] Connection header received
0806 21:07:15.910761 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:15.910764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:15.910819 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:15.910901 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:15.911592 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:15.912457 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:15.914035 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:15.914376 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:15.914465 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:15.918517 (+ 4052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.195:40037: BlockingRecv error: recv got EOF from 127.27.225.195:40037 (error 108)
Metrics: {"server-negotiator.queue_time_us":141}
Aug 06 21:07:16 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:16 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:16.528311 5483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:16.522194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.193:49989)
0806 21:07:16.522488 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:16.522501 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:16.522627 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:16.522900 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:16.522902 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:16.523099 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:16.523198 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:16.523204 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:16.523990 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:16.523994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:16.524568 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:16.524578 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:16.525053 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:16.525057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:16.525145 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:16.525679 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:16.525695 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:16.528081 (+ 2386us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":91,"threads_started":1}
W20260806 21:07:16.528394 5484 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:16.522326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:49989 (local address 127.27.225.254:40811)
0806 21:07:16.522560 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:16.522563 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:16.522577 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:07:16.522729 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:16.522731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:16.522777 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:16.522857 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:16.523322 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:16.523864 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:16.524694 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:16.524919 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:16.525003 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:16.528242 (+ 3239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.193:49989: BlockingRecv error: recv got EOF from 127.27.225.193:49989 (error 108)
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":60,"threads_started":1}
Aug 06 21:07:16 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:16 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:16.720367 5485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:16.714115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.194:55781)
0806 21:07:16.714400 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:16.714423 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:16.714521 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:16.714794 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:16.714796 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:16.714989 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:16.715089 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:16.715095 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:16.715957 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:16.715960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:16.716529 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:16.716538 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:16.717061 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:16.717064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:16.717132 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:16.717884 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:16.717909 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:16.720186 (+ 2277us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":96,"threads_started":1}
W20260806 21:07:16.720463 5484 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:16.714219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:55781 (local address 127.27.225.254:40811)
0806 21:07:16.714398 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:16.714402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:16.714498 (+ 96us) server_negotiation.cc:407] Connection header received
0806 21:07:16.714593 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:16.714595 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:16.714651 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:16.714733 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:16.715241 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:16.715832 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:16.716708 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:16.716938 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:16.717034 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:16.720325 (+ 3291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.194:55781: BlockingRecv error: recv got EOF from 127.27.225.194:55781 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 06 21:07:16 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:16 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:16.925383 5486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:16.919308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.195:58803)
0806 21:07:16.919632 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:16.919651 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:16.919768 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:16.920116 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:16.920119 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:16.920337 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:16.920432 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:16.920438 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:16.921320 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:16.921324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:16.922007 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:16.922017 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:16.922587 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:16.922590 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:16.922676 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:16.923267 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:16.923279 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:16.925263 (+ 1984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":112,"threads_started":1}
W20260806 21:07:16.925591 5484 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:16.919527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:58803 (local address 127.27.225.254:40811)
0806 21:07:16.919726 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:16.919731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:16.919746 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:07:16.919892 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:16.919895 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:16.919998 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:16.920076 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:16.920556 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:16.921201 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:16.922134 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:16.922477 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:16.922581 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:16.925452 (+ 2871us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.195:58803: BlockingRecv error: recv got EOF from 127.27.225.195:58803 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 06 21:07:17 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:17 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:17.535086 5487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:17.529233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.193:48333)
0806 21:07:17.529522 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:17.529536 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:17.529627 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:17.529896 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:17.529898 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:17.530084 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:17.530179 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:17.530185 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:17.530989 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:17.530991 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:17.531563 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:17.531572 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:17.532010 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:17.532012 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:17.532074 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:17.532558 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:17.532569 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:17.534911 (+ 2342us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":109,"threads_started":1}
W20260806 21:07:17.535197 5488 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:17.529333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:48333 (local address 127.27.225.254:40811)
0806 21:07:17.529601 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:17.529605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:17.529620 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:07:17.529724 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:17.529726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:17.529771 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:17.529885 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:17.530316 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:17.530876 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:17.531679 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:17.531906 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:17.531989 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:17.535051 (+ 3062us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.193:48333: BlockingRecv error: recv got EOF from 127.27.225.193:48333 (error 108)
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":108,"threads_started":1}
Aug 06 21:07:17 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:17 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:17.727300 5489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:17.721363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.194:48277)
0806 21:07:17.721630 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:17.721646 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:17.721777 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:17.722057 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:17.722059 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:17.722261 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:17.722355 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:17.722360 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:17.723112 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:17.723114 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:17.723658 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:17.723665 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:17.724118 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:17.724121 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:17.724184 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:17.724764 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:17.724778 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:17.727180 (+ 2402us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":93,"threads_started":1}
W20260806 21:07:17.727525 5488 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:17.721429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:48277 (local address 127.27.225.254:40811)
0806 21:07:17.721683 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:17.721686 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:17.721730 (+ 44us) server_negotiation.cc:407] Connection header received
0806 21:07:17.721892 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:17.721895 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:17.721941 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:17.722022 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:17.722461 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:17.723006 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:17.723786 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:17.724012 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:17.724138 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:17.727401 (+ 3263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.194:48277: BlockingRecv error: recv got EOF from 127.27.225.194:48277 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 06 21:07:17 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:17 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:17.931820 5490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:17.926295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.195:34523)
0806 21:07:17.926600 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:17.926613 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:17.926715 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:17.926961 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:17.926964 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:17.927150 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:17.927243 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:17.927248 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:17.928000 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:17.928002 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:17.928579 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:17.928588 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:17.929044 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:17.929047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:17.929111 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:17.929726 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:17.929744 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:17.931679 (+ 1935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":125,"threads_started":1}
W20260806 21:07:17.932075 5488 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:17.926332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:34523 (local address 127.27.225.254:40811)
0806 21:07:17.926488 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:17.926493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:17.926745 (+ 252us) server_negotiation.cc:407] Connection header received
0806 21:07:17.926796 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:17.926798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:17.926848 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:17.926925 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:17.927387 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:17.927909 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:17.928692 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:17.928924 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:17.929008 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:17.931942 (+ 2934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.195:34523: BlockingRecv error: recv got EOF from 127.27.225.195:34523 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 21:07:18 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:18 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:18.541465 5491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:18.535990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.193:49081)
0806 21:07:18.536323 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:18.536337 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:18.536431 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:18.536697 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:18.536699 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:18.536886 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:18.536984 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:18.536989 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:18.537764 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:18.537774 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:18.538448 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:18.538458 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:18.538869 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:18.538872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:18.538933 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:18.539509 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:18.539521 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:18.541347 (+ 1826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":102,"threads_started":1}
W20260806 21:07:18.541666 5492 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:18.536058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:49081 (local address 127.27.225.254:40811)
0806 21:07:18.536394 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:18.536399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:18.536412 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:07:18.536519 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:18.536521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:18.536564 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:18.536643 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:18.537087 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:18.537644 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:18.538556 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:18.538771 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:18.538846 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:18.541485 (+ 2639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.193:49081: BlockingRecv error: recv got EOF from 127.27.225.193:49081 (error 108)
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":134,"threads_started":1}
Aug 06 21:07:18 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:18 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:18.734153 5493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:18.728205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.194:55191)
0806 21:07:18.728529 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:18.728543 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:18.728655 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:18.728915 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:18.728917 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:18.729101 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:18.729199 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:18.729204 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:18.730043 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:18.730045 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:18.730595 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:18.730601 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:18.731127 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:18.731129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:18.731187 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:18.731905 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:18.731922 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:18.734029 (+ 2107us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":144,"threads_started":1}
W20260806 21:07:18.734319 5492 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:18.728329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:55191 (local address 127.27.225.254:40811)
0806 21:07:18.728480 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:18.728484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:18.728647 (+ 163us) server_negotiation.cc:407] Connection header received
0806 21:07:18.728737 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:18.728739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:18.728788 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:18.728873 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:18.729316 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:18.729944 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:18.730722 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:18.731004 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:18.731128 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:18.734165 (+ 3037us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.194:55191: BlockingRecv error: recv got EOF from 127.27.225.194:55191 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 21:07:18 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:18 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:18.938616 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:18.932739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.195:55429)
0806 21:07:18.933056 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:18.933070 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:18.933166 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:18.933499 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:18.933501 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:18.933703 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:18.933806 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:18.933882 (+ 76us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:18.934690 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:18.934692 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:18.935247 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:18.935254 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:18.935691 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:18.935694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:18.935755 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:18.936266 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:18.936300 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:18.938471 (+ 2171us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":99,"threads_started":1}
W20260806 21:07:18.938932 5492 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:18.932917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:55429 (local address 127.27.225.254:40811)
0806 21:07:18.933103 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:18.933107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:18.933122 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:07:18.933298 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:18.933301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:18.933351 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:18.933423 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:18.934011 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:18.934571 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:18.935381 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:18.935592 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:18.935670 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:18.938813 (+ 3143us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.195:55429: BlockingRecv error: recv got EOF from 127.27.225.195:55429 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 06 21:07:19 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:19 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:19.548851 5495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:19.542414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.193:53191)
0806 21:07:19.542751 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:19.542771 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:19.542875 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:19.543314 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:19.543317 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:19.543529 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:19.543631 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:19.543636 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:19.544463 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:19.544470 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:19.545184 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:19.545197 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:19.545968 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:19.545972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:19.546049 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:19.546662 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:19.546674 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:19.548727 (+ 2053us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":143,"threads_started":1}
W20260806 21:07:19.549120 5496 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:19.542551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:53191 (local address 127.27.225.254:40811)
0806 21:07:19.542804 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:19.542807 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:19.542826 (+ 19us) server_negotiation.cc:407] Connection header received
0806 21:07:19.542950 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:19.542952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:19.542993 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:19.543083 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:19.543757 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:19.544343 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:19.545353 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:19.545751 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:19.545985 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:19.548986 (+ 3001us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.193:53191: BlockingRecv error: recv got EOF from 127.27.225.193:53191 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":93,"threads_started":1}
Aug 06 21:07:19 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:19 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:19.740912 5497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:19.735067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.194:48479)
0806 21:07:19.735438 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:19.735456 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:19.735594 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:19.735889 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:19.735892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:19.736111 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:19.736205 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:19.736211 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:19.737012 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:19.737017 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:19.737584 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:19.737592 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:19.738150 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:19.738153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:19.738239 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:19.738795 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:19.738809 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:19.740781 (+ 1972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":135,"threads_started":1}
W20260806 21:07:19.741106 5496 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:19.735289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:48479 (local address 127.27.225.254:40811)
0806 21:07:19.735462 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:19.735468 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:19.735545 (+ 77us) server_negotiation.cc:407] Connection header received
0806 21:07:19.735708 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:19.735711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:19.735766 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:19.735863 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:19.736358 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:19.736897 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:19.737710 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:19.738010 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:19.738148 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:19.740939 (+ 2791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.194:48479: BlockingRecv error: recv got EOF from 127.27.225.194:48479 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 06 21:07:19 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:19 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:19.945487 5499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:19.939561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.195:39767)
0806 21:07:19.939846 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:19.939866 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:19.940017 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:19.940317 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:19.940319 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:19.940512 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:19.940607 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:19.940612 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:19.941369 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:19.941372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:19.942011 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:19.942021 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:19.942563 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:19.942567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:19.942650 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:19.943239 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:19.943256 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:19.945314 (+ 2058us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":74,"threads_started":1}
W20260806 21:07:19.945562 5496 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:19.939773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:39767 (local address 127.27.225.254:40811)
0806 21:07:19.939944 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:19.939949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:19.939967 (+ 18us) server_negotiation.cc:407] Connection header received
0806 21:07:19.940140 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:19.940143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:19.940201 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:19.940270 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:19.940719 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:19.941241 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:19.942118 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:19.942416 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:19.942525 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:19.945449 (+ 2924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.195:39767: BlockingRecv error: recv got EOF from 127.27.225.195:39767 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 06 21:07:20 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:20 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:20.562520 5500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:20.549711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.193:49851)
0806 21:07:20.550110 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:20.550124 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:20.550223 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:20.550528 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:20.550531 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:20.550730 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:20.550842 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:20.550848 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:20.551726 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:20.551730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:20.552364 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:20.552377 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:20.552936 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:20.552940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:20.553035 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:20.560083 (+ 7048us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:20.560106 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:20.562385 (+ 2279us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":137,"threads_started":1}
W20260806 21:07:20.562891 5501 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:20.549878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:49851 (local address 127.27.225.254:40811)
0806 21:07:20.550186 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:20.550190 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:20.550205 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:07:20.550296 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:20.550298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:20.550349 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:20.550432 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:20.550971 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:20.551579 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:20.552549 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:20.552789 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:20.552881 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:20.562571 (+ 9690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.193:49851: BlockingRecv error: recv got EOF from 127.27.225.193:49851 (error 108)
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":66,"threads_started":1}
Aug 06 21:07:20 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:20 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:20.748194 5502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:20.741798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.194:35607)
0806 21:07:20.742136 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:20.742154 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:20.742274 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:20.742568 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:20.742595 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:20.742787 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:20.742888 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:20.742893 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:20.743787 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:20.743791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:20.744445 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:20.744455 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:20.745025 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:20.745030 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:20.745126 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:20.745745 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:20.745758 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:20.748009 (+ 2251us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":89,"threads_started":1}
W20260806 21:07:20.748515 5501 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:20.741891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:35607 (local address 127.27.225.254:40811)
0806 21:07:20.742042 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:20.742046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:20.742272 (+ 226us) server_negotiation.cc:407] Connection header received
0806 21:07:20.742367 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:20.742370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:20.742443 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:20.742526 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:20.743033 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:20.743648 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:20.744609 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:20.744873 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:20.744975 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:20.748338 (+ 3363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.194:35607: BlockingRecv error: recv got EOF from 127.27.225.194:35607 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 21:07:20 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:20 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.195@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:20.952207 5503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:20.946426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.195:48651)
0806 21:07:20.946713 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:20.946726 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:20.946883 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:20.947180 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:20.947184 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:20.947457 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:20.947564 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:20.947570 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:20.948478 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:20.948481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:20.949058 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:20.949067 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:20.949531 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:20.949533 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:20.949598 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:20.950120 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:20.950132 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:20.952077 (+ 1945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":100,"threads_started":1}
W20260806 21:07:20.952365 5501 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:20.946474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:48651 (local address 127.27.225.254:40811)
0806 21:07:20.946634 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:20.946640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:20.946814 (+ 174us) server_negotiation.cc:407] Connection header received
0806 21:07:20.946989 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:20.946992 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:20.947052 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:20.947135 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:20.947724 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:20.948364 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:20.949185 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:20.949424 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:20.949516 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:20.952215 (+ 2699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.195:48651: BlockingRecv error: recv got EOF from 127.27.225.195:48651 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 21:07:21 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:21 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.193@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:21.569801 5504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:21.563509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.193:39003)
0806 21:07:21.563936 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:21.563950 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:21.564042 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:21.564307 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:21.564309 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:21.564531 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:21.564634 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:21.564640 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:21.565458 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:21.565461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:21.566081 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:21.566093 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:21.566684 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:21.566691 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:21.566793 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:21.567435 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:21.567448 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:21.569627 (+ 2179us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":88,"threads_started":1}
W20260806 21:07:21.570119 5505 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:21.563735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:39003 (local address 127.27.225.254:40811)
0806 21:07:21.563985 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:21.563988 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:21.564002 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:07:21.564135 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:21.564137 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:21.564179 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:21.564254 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:21.564753 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:21.565329 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:21.566199 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:21.566470 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:21.566565 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:21.569976 (+ 3411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.193:39003: BlockingRecv error: recv got EOF from 127.27.225.193:39003 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":94,"threads_started":1}
Aug 06 21:07:21 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:07:21 dist-test-slave-69rg krb5kdc[4427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.225.194@KRBTEST.COM for oryx/127.27.225.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:07:21.755625 5506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:21.749379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40811 (local address 127.27.225.194:46925)
0806 21:07:21.749782 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:21.749795 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:21.749911 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:21.750196 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:21.750199 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:21.750383 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:21.750481 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:21.750486 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:21.751327 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:21.751330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:21.751910 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:21.751918 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:21.752462 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:21.752465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:21.752536 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:21.753168 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:21.753180 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:21.755386 (+ 2206us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.225.254:40811: Server oryx/127.27.225.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":126,"threads_started":1}
W20260806 21:07:21.755669 5505 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:21.749538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:46925 (local address 127.27.225.254:40811)
0806 21:07:21.749724 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:21.749728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:21.749934 (+ 206us) server_negotiation.cc:407] Connection header received
0806 21:07:21.749993 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:21.749996 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:21.750047 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:21.750158 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:21.750642 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:21.751210 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:21.752047 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:21.752312 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:21.752420 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:21.755538 (+ 3118us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.225.194:46925: BlockingRecv error: recv got EOF from 127.27.225.194:46925 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 21:07:21.839859 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 5021
I20260806 21:07:21.848272 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 5176
I20260806 21:07:21.854578 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 5310
I20260806 21:07:21.860983 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 4949
2026-08-06T21:07:21Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18176 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T21:07:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:07:21Z Disabled control of system clock
I20260806 21:07:21.895401 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:46341
--webserver_interface=127.27.225.254
--webserver_port=0
--builtin_ntp_servers=127.27.225.212:39287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:46341
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 21:07:22.012596 5510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:07:22.012905 5510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:07:22.012943 5510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:07:22.017146 5510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:07:22.017230 5510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:07:22.017246 5510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:07:22.017268 5510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:07:22.017297 5510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:07:22.022540 5510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:39287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:46341
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:46341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:07:22.023886 5510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:07:22.025381 5510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:07:22.034425 5516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:22.034440 5518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:22.034468 5510 server_base.cc:1034] running on GCE node
W20260806 21:07:22.034425 5515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:22.035283 5510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:07:22.036419 5510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:07:22.037618 5510 hybrid_clock.cc:648] HybridClock initialized: now 1786050442037592 us; error 42 us; skew 500 ppm
I20260806 21:07:22.039717 5510 webserver.cc:460] Webserver started at http://127.27.225.254:35351/ using document root <none> and password file <none>
I20260806 21:07:22.040323 5510 fs_manager.cc:359] Metadata directory not provided
I20260806 21:07:22.040421 5510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:07:22.040666 5510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:07:22.042999 5510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "1188b9a8aa7c49cb89623e52cbd73afb"
format_stamp: "Formatted at 2026-08-06 21:07:22 on dist-test-slave-69rg"
server_key: "d5d66f7e9153584f2f3073def459043e"
server_key_iv: "f882ebece9c03e8202d60cc2a9a2bbe3"
server_key_version: "encryptionkey@0"
I20260806 21:07:22.043625 5510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "1188b9a8aa7c49cb89623e52cbd73afb"
format_stamp: "Formatted at 2026-08-06 21:07:22 on dist-test-slave-69rg"
server_key: "d5d66f7e9153584f2f3073def459043e"
server_key_iv: "f882ebece9c03e8202d60cc2a9a2bbe3"
server_key_version: "encryptionkey@0"
I20260806 21:07:22.047483 5510 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 21:07:22.050094 5524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:22.051383 5510 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 21:07:22.051534 5510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "1188b9a8aa7c49cb89623e52cbd73afb"
format_stamp: "Formatted at 2026-08-06 21:07:22 on dist-test-slave-69rg"
server_key: "d5d66f7e9153584f2f3073def459043e"
server_key_iv: "f882ebece9c03e8202d60cc2a9a2bbe3"
server_key_version: "encryptionkey@0"
I20260806 21:07:22.051673 5510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:07:22.062773 5510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:07:22.063773 5510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:07:22.064003 5510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:07:22.072081 5510 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:46341
I20260806 21:07:22.072117 5576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:46341 every 8 connection(s)
I20260806 21:07:22.073153 5510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:07:22.076632 5577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:07:22.082471 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 5510
I20260806 21:07:22.082607 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:07:22.082937 28551 external_mini_cluster.cc:1428] Setting key fffc4554bb797265051a59f4de732e14
I20260806 21:07:22.082924 5577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1188b9a8aa7c49cb89623e52cbd73afb: Bootstrap starting.
I20260806 21:07:22.085883 5577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1188b9a8aa7c49cb89623e52cbd73afb: Neither blocks nor log segments found. Creating new log.
I20260806 21:07:22.086747 5577 log.cc:824] T 00000000000000000000000000000000 P 1188b9a8aa7c49cb89623e52cbd73afb: Log is configured to *not* fsync() on all Append() calls
I20260806 21:07:22.089272 5577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1188b9a8aa7c49cb89623e52cbd73afb: No bootstrap required, opened a new log
I20260806 21:07:22.093364 5577 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1188b9a8aa7c49cb89623e52cbd73afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1188b9a8aa7c49cb89623e52cbd73afb" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 46341 } }
I20260806 21:07:22.093633 5577 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1188b9a8aa7c49cb89623e52cbd73afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:07:22.093719 5577 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1188b9a8aa7c49cb89623e52cbd73afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1188b9a8aa7c49cb89623e52cbd73afb, State: Initialized, Role: FOLLOWER
I20260806 21:07:22.093812 5580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:22.084491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33616 (local address 127.27.225.254:46341)
0806 21:07:22.085335 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:22.085352 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:22.085393 (+ 41us) server_negotiation.cc:407] Connection header received
0806 21:07:22.086244 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:22.086267 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:22.086740 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:22.087166 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:22.087847 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:22.088925 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:22.090117 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:22.090656 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:22.090857 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:22.091823 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:22.091829 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:22.091843 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:07:22.092266 (+ 423us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:22.092337 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:07:22.092359 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:22.092478 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:22.092656 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 21:07:22.092948 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":153,"threads_started":1}
I20260806 21:07:22.094285 5577 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1188b9a8aa7c49cb89623e52cbd73afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1188b9a8aa7c49cb89623e52cbd73afb" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 46341 } }
I20260806 21:07:22.094455 5577 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1188b9a8aa7c49cb89623e52cbd73afb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:07:22.094523 5577 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1188b9a8aa7c49cb89623e52cbd73afb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:07:22.094616 5577 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1188b9a8aa7c49cb89623e52cbd73afb [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:07:22.095695 5577 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1188b9a8aa7c49cb89623e52cbd73afb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1188b9a8aa7c49cb89623e52cbd73afb" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 46341 } }
I20260806 21:07:22.096078 5577 leader_election.cc:302] T 00000000000000000000000000000000 P 1188b9a8aa7c49cb89623e52cbd73afb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1188b9a8aa7c49cb89623e52cbd73afb; no voters:
I20260806 21:07:22.096484 5577 leader_election.cc:288] T 00000000000000000000000000000000 P 1188b9a8aa7c49cb89623e52cbd73afb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:07:22.096649 5582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1188b9a8aa7c49cb89623e52cbd73afb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:07:22.096951 5582 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1188b9a8aa7c49cb89623e52cbd73afb [term 1 LEADER]: Becoming Leader. State: Replica: 1188b9a8aa7c49cb89623e52cbd73afb, State: Running, Role: LEADER
I20260806 21:07:22.097281 5582 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1188b9a8aa7c49cb89623e52cbd73afb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1188b9a8aa7c49cb89623e52cbd73afb" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 46341 } }
I20260806 21:07:22.097790 5577 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1188b9a8aa7c49cb89623e52cbd73afb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:07:22.099179 5584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1188b9a8aa7c49cb89623e52cbd73afb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1188b9a8aa7c49cb89623e52cbd73afb. Latest consensus state: current_term: 1 leader_uuid: "1188b9a8aa7c49cb89623e52cbd73afb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1188b9a8aa7c49cb89623e52cbd73afb" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 46341 } } }
I20260806 21:07:22.099210 5583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1188b9a8aa7c49cb89623e52cbd73afb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1188b9a8aa7c49cb89623e52cbd73afb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1188b9a8aa7c49cb89623e52cbd73afb" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 46341 } } }
I20260806 21:07:22.099336 5584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1188b9a8aa7c49cb89623e52cbd73afb [sys.catalog]: This master's current role is: LEADER
I20260806 21:07:22.099336 5583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1188b9a8aa7c49cb89623e52cbd73afb [sys.catalog]: This master's current role is: LEADER
I20260806 21:07:22.099943 5594 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:07:22.104640 5594 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:07:22.112262 5594 catalog_manager.cc:1347] Generated new cluster ID: 79e48208d7f54530bd4293453e2660bb
I20260806 21:07:22.112370 5594 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:07:22.131517 5594 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:07:22.132567 5594 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:07:22.136863 5594 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1188b9a8aa7c49cb89623e52cbd73afb: Generated new TSK 0
I20260806 21:07:22.137678 5594 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:07:22.143972 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:46341
--builtin_ntp_servers=127.27.225.212:39287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 21:07:22.287225 5602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:07:22.287539 5602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:07:22.287580 5602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:07:22.291862 5602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:07:22.291935 5602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:07:22.291992 5602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:07:22.297101 5602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:39287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:46341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:07:22.298549 5602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:07:22.300249 5602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:07:22.310729 5607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:22.310712 5611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:22.310745 5609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:22.311132 5602 server_base.cc:1034] running on GCE node
I20260806 21:07:22.311574 5602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:07:22.312264 5602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:07:22.313443 5602 hybrid_clock.cc:648] HybridClock initialized: now 1786050442313426 us; error 32 us; skew 500 ppm
I20260806 21:07:22.315531 5602 webserver.cc:460] Webserver started at http://127.27.225.193:43433/ using document root <none> and password file <none>
I20260806 21:07:22.316316 5602 fs_manager.cc:359] Metadata directory not provided
I20260806 21:07:22.316425 5602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:07:22.316771 5602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:07:22.319042 5602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "ae6eee2400134634aa0f280fefc793f0"
format_stamp: "Formatted at 2026-08-06 21:07:22 on dist-test-slave-69rg"
server_key: "b01eac0672d06ccd9b2890189ef3dd06"
server_key_iv: "b9483fdf3e8dd541270acd7729d4907a"
server_key_version: "encryptionkey@0"
I20260806 21:07:22.319600 5602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "ae6eee2400134634aa0f280fefc793f0"
format_stamp: "Formatted at 2026-08-06 21:07:22 on dist-test-slave-69rg"
server_key: "b01eac0672d06ccd9b2890189ef3dd06"
server_key_iv: "b9483fdf3e8dd541270acd7729d4907a"
server_key_version: "encryptionkey@0"
I20260806 21:07:22.323622 5602 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 21:07:22.326450 5617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:22.327973 5602 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:07:22.328166 5602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "ae6eee2400134634aa0f280fefc793f0"
format_stamp: "Formatted at 2026-08-06 21:07:22 on dist-test-slave-69rg"
server_key: "b01eac0672d06ccd9b2890189ef3dd06"
server_key_iv: "b9483fdf3e8dd541270acd7729d4907a"
server_key_version: "encryptionkey@0"
I20260806 21:07:22.328287 5602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:07:22.346554 5602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:07:22.347309 5602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:07:22.347502 5602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:07:22.348052 5602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:07:22.348979 5602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:07:22.349030 5602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:22.349078 5602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:07:22.349094 5602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:22.359138 5602 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:38035
I20260806 21:07:22.359177 5732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:38035 every 8 connection(s)
I20260806 21:07:22.360101 5602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:07:22.363840 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 5602
I20260806 21:07:22.363973 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:07:22.364367 28551 external_mini_cluster.cc:1428] Setting key 9a34862c58fa46e7b102ba32b4d9f72c
I20260806 21:07:22.366652 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:46341
--builtin_ntp_servers=127.27.225.212:39287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 21:07:22.370937 5580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:22.362073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:41749 (local address 127.27.225.254:46341)
0806 21:07:22.362267 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:22.362271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:22.363410 (+ 1139us) server_negotiation.cc:407] Connection header received
0806 21:07:22.365243 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:22.365246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:22.365297 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:22.365405 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:22.366575 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:22.367262 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:22.368590 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:22.368862 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:22.368995 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:22.370355 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:22.370359 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:22.370362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:07:22.370378 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:22.370408 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:07:22.370416 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:22.370504 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:22.370707 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 21:07:22.370779 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260806 21:07:22.371912 5735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:22.362496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:46341 (local address 127.27.225.193:41749)
0806 21:07:22.363222 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:22.363267 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:22.364971 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:22.365652 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:22.365663 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:22.365788 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:22.366412 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:22.366427 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:22.367427 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:22.367431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:22.368435 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:22.368445 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:22.369053 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:22.369058 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:22.369258 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:22.370043 (+ 785us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:07:22.370044 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:22.370093 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:07:22.370174 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:07:22.370520 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:07:22.370535 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:07:22.370547 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 21:07:22.370688 (+ 141us) client_negotiation.cc:240] Negotiation successful
0806 21:07:22.371017 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":121,"threads_started":1}
I20260806 21:07:22.373347 5733 heartbeater.cc:347] Connected to a master server at 127.27.225.254:46341
I20260806 21:07:22.373664 5733 heartbeater.cc:464] Registering TS with master...
I20260806 21:07:22.374454 5733 heartbeater.cc:511] Master 127.27.225.254:46341 requested a full tablet report, sending...
I20260806 21:07:22.376462 5541 ts_manager.cc:194] Registered new tserver with Master: ae6eee2400134634aa0f280fefc793f0 (127.27.225.193:38035)
I20260806 21:07:22.378548 5541 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.225.193:41749
W20260806 21:07:22.489748 5736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:07:22.490037 5736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:07:22.490084 5736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:07:22.494094 5736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:07:22.494145 5736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:07:22.494196 5736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:07:22.499321 5736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:39287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:46341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:07:22.500444 5736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:07:22.501809 5736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:07:22.512487 5744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:22.512491 5741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:22.512534 5736 server_base.cc:1034] running on GCE node
W20260806 21:07:22.512487 5742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:22.513314 5736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:07:22.514127 5736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:07:22.515326 5736 hybrid_clock.cc:648] HybridClock initialized: now 1786050442515309 us; error 29 us; skew 500 ppm
I20260806 21:07:22.517339 5736 webserver.cc:460] Webserver started at http://127.27.225.194:36109/ using document root <none> and password file <none>
I20260806 21:07:22.518018 5736 fs_manager.cc:359] Metadata directory not provided
I20260806 21:07:22.518121 5736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:07:22.518370 5736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:07:22.520488 5736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "a1e24aeb72434f0aa67c1f06b8fc70ac"
format_stamp: "Formatted at 2026-08-06 21:07:22 on dist-test-slave-69rg"
server_key: "9285f60d0055c3f1239dc5f8bc90d9c9"
server_key_iv: "f8d86b736ed43d8c73006d86ac3cecbd"
server_key_version: "encryptionkey@0"
I20260806 21:07:22.521075 5736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "a1e24aeb72434f0aa67c1f06b8fc70ac"
format_stamp: "Formatted at 2026-08-06 21:07:22 on dist-test-slave-69rg"
server_key: "9285f60d0055c3f1239dc5f8bc90d9c9"
server_key_iv: "f8d86b736ed43d8c73006d86ac3cecbd"
server_key_version: "encryptionkey@0"
I20260806 21:07:22.525146 5736 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260806 21:07:22.528030 5750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:22.529373 5736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:07:22.529546 5736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "a1e24aeb72434f0aa67c1f06b8fc70ac"
format_stamp: "Formatted at 2026-08-06 21:07:22 on dist-test-slave-69rg"
server_key: "9285f60d0055c3f1239dc5f8bc90d9c9"
server_key_iv: "f8d86b736ed43d8c73006d86ac3cecbd"
server_key_version: "encryptionkey@0"
I20260806 21:07:22.529718 5736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:07:22.546690 5736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:07:22.547435 5736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:07:22.547595 5736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:07:22.548144 5736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:07:22.549067 5736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:07:22.549124 5736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:22.549163 5736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:07:22.549180 5736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:22.558343 5736 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:42255
I20260806 21:07:22.558434 5865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:42255 every 8 connection(s)
I20260806 21:07:22.559345 5736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:07:22.565037 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 5736
I20260806 21:07:22.565189 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:07:22.565491 28551 external_mini_cluster.cc:1428] Setting key b8afdc272a7fe9db09b7efd296baf3e3
I20260806 21:07:22.567590 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:46341
--builtin_ntp_servers=127.27.225.212:39287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 21:07:22.569314 5580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:22.561326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:55767 (local address 127.27.225.254:46341)
0806 21:07:22.561552 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:22.561556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:22.562652 (+ 1096us) server_negotiation.cc:407] Connection header received
0806 21:07:22.564066 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:22.564070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:22.564129 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:22.564221 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:22.565177 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:22.565913 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:22.567309 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:22.567540 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:22.567663 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:22.568803 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:22.568807 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:22.568810 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:07:22.568830 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:22.568848 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:07:22.568856 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:22.568937 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:22.569103 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 21:07:22.569169 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260806 21:07:22.570401 5868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:22.561739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:46341 (local address 127.27.225.194:55767)
0806 21:07:22.562481 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:22.562524 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:22.563850 (+ 1326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:22.564404 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:22.564411 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:22.564495 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:22.565021 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:22.565034 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:22.566066 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:22.566073 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:22.567170 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:22.567180 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:22.567702 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:22.567706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:22.567877 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:22.568529 (+ 652us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:07:22.568531 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:22.568574 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:07:22.568642 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:07:22.568939 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:07:22.568951 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:07:22.568960 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 21:07:22.569111 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 21:07:22.569427 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":110,"threads_started":1}
I20260806 21:07:22.571682 5866 heartbeater.cc:347] Connected to a master server at 127.27.225.254:46341
I20260806 21:07:22.571949 5866 heartbeater.cc:464] Registering TS with master...
I20260806 21:07:22.572695 5866 heartbeater.cc:511] Master 127.27.225.254:46341 requested a full tablet report, sending...
I20260806 21:07:22.574023 5541 ts_manager.cc:194] Registered new tserver with Master: a1e24aeb72434f0aa67c1f06b8fc70ac (127.27.225.194:42255)
I20260806 21:07:22.574626 5541 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.225.194:55767
W20260806 21:07:22.689427 5869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:07:22.689706 5869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:07:22.689745 5869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:07:22.693930 5869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:07:22.693996 5869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:07:22.694099 5869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:07:22.699119 5869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:39287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:46341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:07:22.700307 5869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:07:22.701723 5869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:07:22.711521 5877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:22.711547 5875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:22.711603 5874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:22.711776 5869 server_base.cc:1034] running on GCE node
I20260806 21:07:22.712231 5869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:07:22.712899 5869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:07:22.714067 5869 hybrid_clock.cc:648] HybridClock initialized: now 1786050442714030 us; error 47 us; skew 500 ppm
I20260806 21:07:22.716051 5869 webserver.cc:460] Webserver started at http://127.27.225.195:42543/ using document root <none> and password file <none>
I20260806 21:07:22.716634 5869 fs_manager.cc:359] Metadata directory not provided
I20260806 21:07:22.716699 5869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:07:22.716868 5869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:07:22.719082 5869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "17690d947cc449b4b7e6a3c60cdf1a6a"
format_stamp: "Formatted at 2026-08-06 21:07:22 on dist-test-slave-69rg"
server_key: "409874746be3d12ce1f86fb6231e4c1c"
server_key_iv: "f53827bb60b21694648430d401bf49c4"
server_key_version: "encryptionkey@0"
I20260806 21:07:22.719655 5869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "17690d947cc449b4b7e6a3c60cdf1a6a"
format_stamp: "Formatted at 2026-08-06 21:07:22 on dist-test-slave-69rg"
server_key: "409874746be3d12ce1f86fb6231e4c1c"
server_key_iv: "f53827bb60b21694648430d401bf49c4"
server_key_version: "encryptionkey@0"
I20260806 21:07:22.723748 5869 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:07:22.726576 5884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:22.728130 5869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:07:22.728257 5869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "17690d947cc449b4b7e6a3c60cdf1a6a"
format_stamp: "Formatted at 2026-08-06 21:07:22 on dist-test-slave-69rg"
server_key: "409874746be3d12ce1f86fb6231e4c1c"
server_key_iv: "f53827bb60b21694648430d401bf49c4"
server_key_version: "encryptionkey@0"
I20260806 21:07:22.728367 5869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:07:22.752345 5869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:07:22.753038 5869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:07:22.753211 5869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:07:22.753763 5869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:07:22.754779 5869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:07:22.754827 5869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:22.754868 5869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:07:22.754884 5869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:22.765268 5869 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:40511
I20260806 21:07:22.765292 5999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:40511 every 8 connection(s)
I20260806 21:07:22.766412 5869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:07:22.776300 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 5869
I20260806 21:07:22.776353 5580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:22.768392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:44011 (local address 127.27.225.254:46341)
0806 21:07:22.768536 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:22.768540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:22.769684 (+ 1144us) server_negotiation.cc:407] Connection header received
0806 21:07:22.771091 (+ 1407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:22.771095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:22.771144 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:22.771229 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:22.772134 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:22.772698 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:22.774311 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:22.774538 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:22.774621 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:22.775751 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:22.775755 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:22.775758 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:07:22.775772 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:22.775791 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:07:22.775798 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:22.775861 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:22.776071 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 21:07:22.776145 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 21:07:22.776432 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:07:22.776752 28551 external_mini_cluster.cc:1428] Setting key 6ab25e5e41c9fb06cbd2459c09346636
I20260806 21:07:22.777431 6002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:22.768795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:46341 (local address 127.27.225.195:44011)
0806 21:07:22.769492 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:22.769534 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:22.770877 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:22.771397 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:22.771403 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:22.771487 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:22.771998 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:22.772010 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:22.772806 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:22.772808 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:22.774179 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:22.774193 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:22.774691 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:22.774693 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:22.774838 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:22.775479 (+ 641us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:07:22.775481 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:22.775521 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:07:22.775604 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:07:22.775910 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:07:22.775918 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:07:22.775924 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 21:07:22.776044 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 21:07:22.776315 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":128,"threads_started":1}
I20260806 21:07:22.778717 6000 heartbeater.cc:347] Connected to a master server at 127.27.225.254:46341
I20260806 21:07:22.778968 6000 heartbeater.cc:464] Registering TS with master...
I20260806 21:07:22.779501 6000 heartbeater.cc:511] Master 127.27.225.254:46341 requested a full tablet report, sending...
I20260806 21:07:22.780512 5541 ts_manager.cc:194] Registered new tserver with Master: 17690d947cc449b4b7e6a3c60cdf1a6a (127.27.225.195:40511)
I20260806 21:07:22.781097 5541 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.225.195:44011
I20260806 21:07:22.791042 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 21:07:22.794122 5580 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:22.793044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33628 (local address 127.27.225.254:46341)
0806 21:07:22.793234 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:22.793239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:22.793405 (+ 166us) server_negotiation.cc:407] Connection header received
0806 21:07:22.793565 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:22.793568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:22.793627 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:22.793715 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:22.793981 (+ 266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33628: BlockingRecv error: recv got EOF from 127.0.0.1:33628 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 21:07:22.794238 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 5602
I20260806 21:07:22.800650 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 5736
I20260806 21:07:22.806797 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 5869
I20260806 21:07:22.812971 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 5510
2026-08-06T21:07:22Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (950 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T21:07:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:07:22Z Disabled control of system clock
I20260806 21:07:22.845069 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:37157
--webserver_interface=127.27.225.254
--webserver_port=0
--builtin_ntp_servers=127.27.225.212:46497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:37157
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 21:07:22.960654 6011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:07:22.960942 6011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:07:22.960981 6011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:07:22.965094 6011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:07:22.965178 6011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:07:22.965193 6011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:07:22.965209 6011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:07:22.965224 6011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:07:22.970194 6011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:46497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:37157
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:37157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:07:22.971330 6011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:07:22.972703 6011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:07:22.983222 6017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:22.983318 6020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:22.983309 6011 server_base.cc:1034] running on GCE node
W20260806 21:07:22.983197 6018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:22.984375 6011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:07:22.985518 6011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:07:22.986721 6011 hybrid_clock.cc:648] HybridClock initialized: now 1786050442986690 us; error 45 us; skew 500 ppm
I20260806 21:07:22.989070 6011 webserver.cc:460] Webserver started at http://127.27.225.254:40651/ using document root <none> and password file <none>
I20260806 21:07:22.989745 6011 fs_manager.cc:359] Metadata directory not provided
I20260806 21:07:22.989872 6011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:07:22.990120 6011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:07:22.992420 6011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "083e63fba29841eaa6a34c728933463d"
format_stamp: "Formatted at 2026-08-06 21:07:22 on dist-test-slave-69rg"
server_key: "3c3d6328ac83e932ed4a71f987c62b45"
server_key_iv: "bb5801477e0ef586c2c47aa80ce6bb16"
server_key_version: "encryptionkey@0"
I20260806 21:07:22.993029 6011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "083e63fba29841eaa6a34c728933463d"
format_stamp: "Formatted at 2026-08-06 21:07:22 on dist-test-slave-69rg"
server_key: "3c3d6328ac83e932ed4a71f987c62b45"
server_key_iv: "bb5801477e0ef586c2c47aa80ce6bb16"
server_key_version: "encryptionkey@0"
I20260806 21:07:22.997260 6011 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260806 21:07:23.000061 6026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:23.001526 6011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:07:23.001711 6011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "083e63fba29841eaa6a34c728933463d"
format_stamp: "Formatted at 2026-08-06 21:07:22 on dist-test-slave-69rg"
server_key: "3c3d6328ac83e932ed4a71f987c62b45"
server_key_iv: "bb5801477e0ef586c2c47aa80ce6bb16"
server_key_version: "encryptionkey@0"
I20260806 21:07:23.001890 6011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:07:23.007052 6011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:07:23.007932 6011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:07:23.008090 6011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:07:23.017059 6011 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:37157
I20260806 21:07:23.017061 6080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:37157 every 8 connection(s)
I20260806 21:07:23.018451 6011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:07:23.022178 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 6011
I20260806 21:07:23.022295 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:07:23.022614 28551 external_mini_cluster.cc:1428] Setting key 1617490286a9c318c7605bd3adec016f
I20260806 21:07:23.022722 6081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:07:23.029313 6084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:23.024369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40852 (local address 127.27.225.254:37157)
0806 21:07:23.025244 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:23.025264 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:23.025305 (+ 41us) server_negotiation.cc:407] Connection header received
0806 21:07:23.026282 (+ 977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:23.026304 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:23.026706 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:23.027099 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:23.027922 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:23.027927 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:23.027939 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:07:23.027950 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:23.028005 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:07:23.028022 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:23.028205 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:23.028289 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 21:07:23.028554 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":162,"threads_started":1}
I20260806 21:07:23.030032 6081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 083e63fba29841eaa6a34c728933463d: Bootstrap starting.
I20260806 21:07:23.032423 6081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 083e63fba29841eaa6a34c728933463d: Neither blocks nor log segments found. Creating new log.
I20260806 21:07:23.033196 6081 log.cc:824] T 00000000000000000000000000000000 P 083e63fba29841eaa6a34c728933463d: Log is configured to *not* fsync() on all Append() calls
I20260806 21:07:23.035846 6081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 083e63fba29841eaa6a34c728933463d: No bootstrap required, opened a new log
I20260806 21:07:23.039903 6081 raft_consensus.cc:348] T 00000000000000000000000000000000 P 083e63fba29841eaa6a34c728933463d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083e63fba29841eaa6a34c728933463d" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37157 } }
I20260806 21:07:23.040192 6081 raft_consensus.cc:374] T 00000000000000000000000000000000 P 083e63fba29841eaa6a34c728933463d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:07:23.040277 6081 raft_consensus.cc:729] T 00000000000000000000000000000000 P 083e63fba29841eaa6a34c728933463d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 083e63fba29841eaa6a34c728933463d, State: Initialized, Role: FOLLOWER
I20260806 21:07:23.040717 6081 consensus_queue.cc:260] T 00000000000000000000000000000000 P 083e63fba29841eaa6a34c728933463d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083e63fba29841eaa6a34c728933463d" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37157 } }
I20260806 21:07:23.040881 6081 raft_consensus.cc:388] T 00000000000000000000000000000000 P 083e63fba29841eaa6a34c728933463d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:07:23.040971 6081 raft_consensus.cc:482] T 00000000000000000000000000000000 P 083e63fba29841eaa6a34c728933463d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:07:23.041093 6081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 083e63fba29841eaa6a34c728933463d [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:07:23.042275 6081 raft_consensus.cc:504] T 00000000000000000000000000000000 P 083e63fba29841eaa6a34c728933463d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083e63fba29841eaa6a34c728933463d" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37157 } }
I20260806 21:07:23.042725 6081 leader_election.cc:302] T 00000000000000000000000000000000 P 083e63fba29841eaa6a34c728933463d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 083e63fba29841eaa6a34c728933463d; no voters:
I20260806 21:07:23.043190 6081 leader_election.cc:288] T 00000000000000000000000000000000 P 083e63fba29841eaa6a34c728933463d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:07:23.043336 6086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 083e63fba29841eaa6a34c728933463d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:07:23.043553 6086 raft_consensus.cc:686] T 00000000000000000000000000000000 P 083e63fba29841eaa6a34c728933463d [term 1 LEADER]: Becoming Leader. State: Replica: 083e63fba29841eaa6a34c728933463d, State: Running, Role: LEADER
I20260806 21:07:23.043934 6086 consensus_queue.cc:237] T 00000000000000000000000000000000 P 083e63fba29841eaa6a34c728933463d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083e63fba29841eaa6a34c728933463d" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37157 } }
I20260806 21:07:23.044770 6081 sys_catalog.cc:564] T 00000000000000000000000000000000 P 083e63fba29841eaa6a34c728933463d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:07:23.045928 6088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 083e63fba29841eaa6a34c728933463d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 083e63fba29841eaa6a34c728933463d. Latest consensus state: current_term: 1 leader_uuid: "083e63fba29841eaa6a34c728933463d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083e63fba29841eaa6a34c728933463d" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37157 } } }
I20260806 21:07:23.046085 6088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 083e63fba29841eaa6a34c728933463d [sys.catalog]: This master's current role is: LEADER
I20260806 21:07:23.046444 6087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 083e63fba29841eaa6a34c728933463d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "083e63fba29841eaa6a34c728933463d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "083e63fba29841eaa6a34c728933463d" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37157 } } }
I20260806 21:07:23.046581 6087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 083e63fba29841eaa6a34c728933463d [sys.catalog]: This master's current role is: LEADER
I20260806 21:07:23.046491 6095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:07:23.050227 6095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:07:23.056320 6095 catalog_manager.cc:1347] Generated new cluster ID: 171e9c1f23f6414f97506c075918699f
I20260806 21:07:23.056423 6095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:07:23.067315 6095 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:07:23.067700 6095 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:07:23.080046 6095 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 083e63fba29841eaa6a34c728933463d: Generated new TSK 0
I20260806 21:07:23.080892 6095 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:07:23.087390 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:37157
--builtin_ntp_servers=127.27.225.212:46497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 21:07:23.205839 6105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:07:23.206176 6105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:07:23.206262 6105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:07:23.210638 6105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:07:23.210736 6105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:07:23.210829 6105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:07:23.215904 6105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:46497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:37157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:07:23.217113 6105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:07:23.218722 6105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:07:23.229439 6114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:23.229398 6111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:23.229452 6105 server_base.cc:1034] running on GCE node
W20260806 21:07:23.229647 6112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:23.230206 6105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:07:23.230996 6105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:07:23.232213 6105 hybrid_clock.cc:648] HybridClock initialized: now 1786050443232189 us; error 50 us; skew 500 ppm
I20260806 21:07:23.234438 6105 webserver.cc:460] Webserver started at http://127.27.225.193:42895/ using document root <none> and password file <none>
I20260806 21:07:23.235108 6105 fs_manager.cc:359] Metadata directory not provided
I20260806 21:07:23.235214 6105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:07:23.235455 6105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:07:23.237591 6105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "4d5ca95e701548b2936a2c4952569c27"
format_stamp: "Formatted at 2026-08-06 21:07:23 on dist-test-slave-69rg"
server_key: "bf5e07edbf8331b039c3b723b2cd375b"
server_key_iv: "a3e91991c973ca3ac599a7e9c4e5a2da"
server_key_version: "encryptionkey@0"
I20260806 21:07:23.238265 6105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "4d5ca95e701548b2936a2c4952569c27"
format_stamp: "Formatted at 2026-08-06 21:07:23 on dist-test-slave-69rg"
server_key: "bf5e07edbf8331b039c3b723b2cd375b"
server_key_iv: "a3e91991c973ca3ac599a7e9c4e5a2da"
server_key_version: "encryptionkey@0"
I20260806 21:07:23.242517 6105 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 21:07:23.245477 6120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:23.246984 6105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:07:23.247144 6105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "4d5ca95e701548b2936a2c4952569c27"
format_stamp: "Formatted at 2026-08-06 21:07:23 on dist-test-slave-69rg"
server_key: "bf5e07edbf8331b039c3b723b2cd375b"
server_key_iv: "a3e91991c973ca3ac599a7e9c4e5a2da"
server_key_version: "encryptionkey@0"
I20260806 21:07:23.247291 6105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:07:23.252156 6105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:07:23.252846 6105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:07:23.253046 6105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:07:23.253633 6105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:07:23.254645 6105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:07:23.254719 6105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:23.254781 6105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:07:23.254827 6105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:23.264073 6105 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:41527
I20260806 21:07:23.264106 6235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:41527 every 8 connection(s)
I20260806 21:07:23.265041 6105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:07:23.265422 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 6105
I20260806 21:07:23.265547 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:07:23.265952 28551 external_mini_cluster.cc:1428] Setting key 95742dc795a91b9a13e99d0998e71d71
I20260806 21:07:23.268353 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:37157
--builtin_ntp_servers=127.27.225.212:46497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 21:07:23.271828 6084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:23.267159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:35209 (local address 127.27.225.254:37157)
0806 21:07:23.267306 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:23.267311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:23.268666 (+ 1355us) server_negotiation.cc:407] Connection header received
0806 21:07:23.270067 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:23.270071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:23.270114 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:23.270196 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:23.271351 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:23.271355 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:23.271358 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:07:23.271377 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:23.271397 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:07:23.271405 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:23.271499 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:23.271645 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 21:07:23.271691 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 21:07:23.272758 6238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:23.267732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:37157 (local address 127.27.225.193:35209)
0806 21:07:23.268512 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:23.268577 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:23.269813 (+ 1236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:23.270380 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:23.270386 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:23.270469 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:23.271091 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:07:23.271092 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:23.271131 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:07:23.271196 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:07:23.271495 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:07:23.271505 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:07:23.271512 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 21:07:23.271691 (+ 179us) client_negotiation.cc:240] Negotiation successful
0806 21:07:23.271937 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":144,"threads_started":1}
I20260806 21:07:23.274128 6236 heartbeater.cc:347] Connected to a master server at 127.27.225.254:37157
I20260806 21:07:23.274370 6236 heartbeater.cc:464] Registering TS with master...
I20260806 21:07:23.274808 6236 heartbeater.cc:511] Master 127.27.225.254:37157 requested a full tablet report, sending...
I20260806 21:07:23.276357 6045 ts_manager.cc:194] Registered new tserver with Master: 4d5ca95e701548b2936a2c4952569c27 (127.27.225.193:41527)
W20260806 21:07:23.384337 6239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:07:23.384662 6239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:07:23.384738 6239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:07:23.388828 6239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:07:23.388938 6239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:07:23.389035 6239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:07:23.393889 6239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:46497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:37157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:07:23.395062 6239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:07:23.396497 6239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:07:23.406788 6245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:23.406788 6244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:23.406802 6247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:23.407476 6239 server_base.cc:1034] running on GCE node
I20260806 21:07:23.407922 6239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:07:23.408617 6239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:07:23.409864 6239 hybrid_clock.cc:648] HybridClock initialized: now 1786050443409823 us; error 50 us; skew 500 ppm
I20260806 21:07:23.412659 6239 webserver.cc:460] Webserver started at http://127.27.225.194:39153/ using document root <none> and password file <none>
I20260806 21:07:23.413515 6239 fs_manager.cc:359] Metadata directory not provided
I20260806 21:07:23.413635 6239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:07:23.413942 6239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:07:23.416271 6239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "d40046ec26a94dcea9d54d0b9a40174e"
format_stamp: "Formatted at 2026-08-06 21:07:23 on dist-test-slave-69rg"
server_key: "38ead197242942e7c295855340da176f"
server_key_iv: "dc8b522bd196c2a2927b643112e4673e"
server_key_version: "encryptionkey@0"
I20260806 21:07:23.416931 6239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "d40046ec26a94dcea9d54d0b9a40174e"
format_stamp: "Formatted at 2026-08-06 21:07:23 on dist-test-slave-69rg"
server_key: "38ead197242942e7c295855340da176f"
server_key_iv: "dc8b522bd196c2a2927b643112e4673e"
server_key_version: "encryptionkey@0"
I20260806 21:07:23.421149 6239 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:07:23.423952 6253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:23.425292 6239 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:07:23.425441 6239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "d40046ec26a94dcea9d54d0b9a40174e"
format_stamp: "Formatted at 2026-08-06 21:07:23 on dist-test-slave-69rg"
server_key: "38ead197242942e7c295855340da176f"
server_key_iv: "dc8b522bd196c2a2927b643112e4673e"
server_key_version: "encryptionkey@0"
I20260806 21:07:23.425572 6239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:07:23.430328 6239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:07:23.430950 6239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:07:23.431143 6239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:07:23.431701 6239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:07:23.432641 6239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:07:23.432713 6239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:23.432775 6239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:07:23.432823 6239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:23.442111 6239 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:43405
I20260806 21:07:23.442128 6369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:43405 every 8 connection(s)
I20260806 21:07:23.443056 6239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:07:23.446137 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 6239
I20260806 21:07:23.446281 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:07:23.446626 28551 external_mini_cluster.cc:1428] Setting key 12c0fbbd0e0368cde8bfaf796af03d45
I20260806 21:07:23.448887 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:37157
--builtin_ntp_servers=127.27.225.212:46497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 21:07:23.451336 6084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:23.445093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:60367 (local address 127.27.225.254:37157)
0806 21:07:23.445251 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:23.445255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:23.447729 (+ 2474us) server_negotiation.cc:407] Connection header received
0806 21:07:23.449353 (+ 1624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:23.449357 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:23.449401 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:23.449489 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:23.450803 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:23.450807 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:23.450810 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:07:23.450825 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:23.450853 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:07:23.450861 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:23.450968 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:23.451144 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 21:07:23.451193 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 21:07:23.452348 6372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:23.445450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:37157 (local address 127.27.225.194:60367)
0806 21:07:23.447560 (+ 2110us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:23.447622 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:23.449123 (+ 1501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:23.449674 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:23.449681 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:23.449772 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:23.450526 (+ 754us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:07:23.450528 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:23.450573 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:07:23.450645 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:07:23.450955 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:07:23.450964 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:07:23.450970 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 21:07:23.451179 (+ 209us) client_negotiation.cc:240] Negotiation successful
0806 21:07:23.451434 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":123,"threads_started":1}
I20260806 21:07:23.453579 6370 heartbeater.cc:347] Connected to a master server at 127.27.225.254:37157
I20260806 21:07:23.453884 6370 heartbeater.cc:464] Registering TS with master...
I20260806 21:07:23.454339 6370 heartbeater.cc:511] Master 127.27.225.254:37157 requested a full tablet report, sending...
I20260806 21:07:23.455444 6045 ts_manager.cc:194] Registered new tserver with Master: d40046ec26a94dcea9d54d0b9a40174e (127.27.225.194:43405)
W20260806 21:07:23.568099 6373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:07:23.568396 6373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:07:23.568465 6373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:07:23.572538 6373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:07:23.572649 6373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:07:23.572741 6373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:07:23.578487 6373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:46497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:37157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:07:23.579679 6373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:07:23.581097 6373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:07:23.591076 6373 server_base.cc:1034] running on GCE node
W20260806 21:07:23.591022 6378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:23.591020 6379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:23.591037 6381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:23.591701 6373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:07:23.592386 6373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:07:23.593586 6373 hybrid_clock.cc:648] HybridClock initialized: now 1786050443593550 us; error 52 us; skew 500 ppm
I20260806 21:07:23.595577 6373 webserver.cc:460] Webserver started at http://127.27.225.195:39215/ using document root <none> and password file <none>
I20260806 21:07:23.596171 6373 fs_manager.cc:359] Metadata directory not provided
I20260806 21:07:23.596268 6373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:07:23.596503 6373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:07:23.598739 6373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "a1373bff31a94b3f85cdd719bcdd16ab"
format_stamp: "Formatted at 2026-08-06 21:07:23 on dist-test-slave-69rg"
server_key: "e43ee69906e92d225f06922b178aa068"
server_key_iv: "b3477e5e7c5bb39d2251cb84438be59b"
server_key_version: "encryptionkey@0"
I20260806 21:07:23.599344 6373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "a1373bff31a94b3f85cdd719bcdd16ab"
format_stamp: "Formatted at 2026-08-06 21:07:23 on dist-test-slave-69rg"
server_key: "e43ee69906e92d225f06922b178aa068"
server_key_iv: "b3477e5e7c5bb39d2251cb84438be59b"
server_key_version: "encryptionkey@0"
I20260806 21:07:23.603211 6373 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:07:23.605875 6387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:23.607164 6373 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:07:23.607260 6373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "a1373bff31a94b3f85cdd719bcdd16ab"
format_stamp: "Formatted at 2026-08-06 21:07:23 on dist-test-slave-69rg"
server_key: "e43ee69906e92d225f06922b178aa068"
server_key_iv: "b3477e5e7c5bb39d2251cb84438be59b"
server_key_version: "encryptionkey@0"
I20260806 21:07:23.607349 6373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:07:23.612040 6373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:07:23.612613 6373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:07:23.612764 6373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:07:23.613340 6373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:07:23.614341 6373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:07:23.614391 6373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:23.614429 6373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:07:23.614445 6373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:23.623275 6373 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:41875
I20260806 21:07:23.623297 6502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:41875 every 8 connection(s)
I20260806 21:07:23.624235 6373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:07:23.627959 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 6373
I20260806 21:07:23.628055 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:07:23.628396 28551 external_mini_cluster.cc:1428] Setting key ce14ccb32cc30708752cb8013da08a42
I20260806 21:07:23.630985 6084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:23.626223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:59157 (local address 127.27.225.254:37157)
0806 21:07:23.626372 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:23.626376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:23.627460 (+ 1084us) server_negotiation.cc:407] Connection header received
0806 21:07:23.629019 (+ 1559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:23.629023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:23.629067 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:23.629155 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:23.630558 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:23.630562 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:23.630565 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:07:23.630577 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:23.630600 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:07:23.630610 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:23.630720 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:23.630848 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 21:07:23.630886 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 21:07:23.631898 6505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:23.626664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:37157 (local address 127.27.225.195:59157)
0806 21:07:23.627309 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:23.627349 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:23.628793 (+ 1444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:23.629386 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:23.629396 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:23.629513 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:23.630276 (+ 763us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:07:23.630277 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:23.630318 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:07:23.630406 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:07:23.630710 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:07:23.630720 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:07:23.630725 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 21:07:23.630867 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 21:07:23.631058 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":111,"threads_started":1}
I20260806 21:07:23.633013 6503 heartbeater.cc:347] Connected to a master server at 127.27.225.254:37157
I20260806 21:07:23.633270 6503 heartbeater.cc:464] Registering TS with master...
I20260806 21:07:23.633685 6503 heartbeater.cc:511] Master 127.27.225.254:37157 requested a full tablet report, sending...
I20260806 21:07:23.634748 6045 ts_manager.cc:194] Registered new tserver with Master: a1373bff31a94b3f85cdd719bcdd16ab (127.27.225.195:41875)
I20260806 21:07:23.642926 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 21:07:23.646291 6084 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:07:23.645188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40854 (local address 127.27.225.254:37157)
0806 21:07:23.645356 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:23.645367 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:23.645617 (+ 250us) server_negotiation.cc:407] Connection header received
0806 21:07:23.645672 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:23.645675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:23.645712 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:23.645775 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:23.646121 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40854: BlockingRecv error: recv got EOF from 127.0.0.1:40854 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 21:07:23.646354 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 6105
I20260806 21:07:23.652774 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 6239
I20260806 21:07:23.658664 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 6373
I20260806 21:07:23.665072 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 6011
2026-08-06T21:07:23Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (852 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:07:23 dist-test-slave-69rg krb5kdc[6513](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:07:23 dist-test-slave-69rg krb5kdc[6513](info): set up 2 sockets
Aug 06 21:07:23 dist-test-slave-69rg krb5kdc[6513](info): commencing operation
krb5kdc: starting...
W20260806 21:07:24.758625 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:07:24 dist-test-slave-69rg krb5kdc[6513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:07:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:07:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:07:24.902065 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:40693
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:41819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:40693
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:07:25.017481 6526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:07:25.017907 6526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:07:25.018000 6526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:07:25.022399 6526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:07:25.022531 6526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:07:25.022567 6526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:07:25.022619 6526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:07:25.022651 6526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:07:25.028353 6526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:41819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:40693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:40693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:07:25.029577 6526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:07:25.030737 6526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:07:25.040295 6532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:25.040441 6526 server_base.cc:1034] running on GCE node
W20260806 21:07:25.040306 6534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:25.040323 6531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:25.041330 6526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:07:25.042627 6526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:07:25.043823 6526 hybrid_clock.cc:648] HybridClock initialized: now 1786050445043796 us; error 41 us; skew 500 ppm
Aug 06 21:07:25 dist-test-slave-69rg krb5kdc[6513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050445, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:07:25.046963 6526 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:07:25.047991 6526 webserver.cc:460] Webserver started at http://127.27.225.254:39235/ using document root <none> and password file <none>
I20260806 21:07:25.048576 6526 fs_manager.cc:359] Metadata directory not provided
I20260806 21:07:25.048650 6526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:07:25.048884 6526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:07:25.051121 6526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "5a8ad42c82ce4a948db32a55554092e4"
format_stamp: "Formatted at 2026-08-06 21:07:25 on dist-test-slave-69rg"
server_key: "53e825ff33b5064ff5f1e4affda995bb"
server_key_iv: "d3ff20b4c81bf215778099ed1e4d72fc"
server_key_version: "encryptionkey@0"
I20260806 21:07:25.051900 6526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "5a8ad42c82ce4a948db32a55554092e4"
format_stamp: "Formatted at 2026-08-06 21:07:25 on dist-test-slave-69rg"
server_key: "53e825ff33b5064ff5f1e4affda995bb"
server_key_iv: "d3ff20b4c81bf215778099ed1e4d72fc"
server_key_version: "encryptionkey@0"
I20260806 21:07:25.056274 6526 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:07:25.059217 6541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:25.060611 6526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:07:25.060801 6526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "5a8ad42c82ce4a948db32a55554092e4"
format_stamp: "Formatted at 2026-08-06 21:07:25 on dist-test-slave-69rg"
server_key: "53e825ff33b5064ff5f1e4affda995bb"
server_key_iv: "d3ff20b4c81bf215778099ed1e4d72fc"
server_key_version: "encryptionkey@0"
I20260806 21:07:25.060941 6526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:07:25.072319 6526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:07:25.076064 6526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:07:25.076320 6526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:07:25.084645 6526 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:40693
I20260806 21:07:25.084697 6593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:40693 every 8 connection(s)
I20260806 21:07:25.085673 6526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:07:25.088693 6594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:07:25.089895 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 6526
I20260806 21:07:25.090025 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:07:25.090376 28551 external_mini_cluster.cc:1428] Setting key 79c20fd5199f2c65dfdbce85d783bf91
I20260806 21:07:25.095954 6594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a8ad42c82ce4a948db32a55554092e4: Bootstrap starting.
I20260806 21:07:25.098599 6594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a8ad42c82ce4a948db32a55554092e4: Neither blocks nor log segments found. Creating new log.
I20260806 21:07:25.099417 6594 log.cc:824] T 00000000000000000000000000000000 P 5a8ad42c82ce4a948db32a55554092e4: Log is configured to *not* fsync() on all Append() calls
Aug 06 21:07:25 dist-test-slave-69rg krb5kdc[6513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:07:25.102229 6594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a8ad42c82ce4a948db32a55554092e4: No bootstrap required, opened a new log
I20260806 21:07:25.106619 6597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:25.091887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53170 (local address 127.27.225.254:40693)
0806 21:07:25.092351 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:25.092383 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:25.092419 (+ 36us) server_negotiation.cc:407] Connection header received
0806 21:07:25.093055 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:25.093071 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:25.093447 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:25.093752 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:25.094720 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.096022 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.097098 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.097618 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.097849 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:25.100730 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:25.100738 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:25.100758 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:07:25.100789 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:25.103127 (+ 2338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:25.103671 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:25.103677 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:25.103683 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:25.103839 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:25.104125 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:25.104129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:25.104130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:25.104721 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:25.104962 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:25.105178 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 21:07:25.105496 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":153,"threads_started":1}
I20260806 21:07:25.107165 6594 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a8ad42c82ce4a948db32a55554092e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a8ad42c82ce4a948db32a55554092e4" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40693 } }
I20260806 21:07:25.107448 6594 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a8ad42c82ce4a948db32a55554092e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:07:25.107569 6594 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a8ad42c82ce4a948db32a55554092e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a8ad42c82ce4a948db32a55554092e4, State: Initialized, Role: FOLLOWER
I20260806 21:07:25.108078 6594 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a8ad42c82ce4a948db32a55554092e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a8ad42c82ce4a948db32a55554092e4" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40693 } }
I20260806 21:07:25.108207 6594 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a8ad42c82ce4a948db32a55554092e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:07:25.108299 6594 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a8ad42c82ce4a948db32a55554092e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:07:25.108397 6594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a8ad42c82ce4a948db32a55554092e4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:07:25.109575 6594 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a8ad42c82ce4a948db32a55554092e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a8ad42c82ce4a948db32a55554092e4" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40693 } }
I20260806 21:07:25.110029 6594 leader_election.cc:302] T 00000000000000000000000000000000 P 5a8ad42c82ce4a948db32a55554092e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a8ad42c82ce4a948db32a55554092e4; no voters:
I20260806 21:07:25.110415 6594 leader_election.cc:288] T 00000000000000000000000000000000 P 5a8ad42c82ce4a948db32a55554092e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:07:25.110618 6599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a8ad42c82ce4a948db32a55554092e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:07:25.110903 6599 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a8ad42c82ce4a948db32a55554092e4 [term 1 LEADER]: Becoming Leader. State: Replica: 5a8ad42c82ce4a948db32a55554092e4, State: Running, Role: LEADER
I20260806 21:07:25.111265 6599 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a8ad42c82ce4a948db32a55554092e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a8ad42c82ce4a948db32a55554092e4" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40693 } }
I20260806 21:07:25.111739 6594 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a8ad42c82ce4a948db32a55554092e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:07:25.113117 6600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a8ad42c82ce4a948db32a55554092e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a8ad42c82ce4a948db32a55554092e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a8ad42c82ce4a948db32a55554092e4" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40693 } } }
I20260806 21:07:25.113257 6600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a8ad42c82ce4a948db32a55554092e4 [sys.catalog]: This master's current role is: LEADER
I20260806 21:07:25.113669 6608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:07:25.113616 6601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a8ad42c82ce4a948db32a55554092e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a8ad42c82ce4a948db32a55554092e4. Latest consensus state: current_term: 1 leader_uuid: "5a8ad42c82ce4a948db32a55554092e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a8ad42c82ce4a948db32a55554092e4" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40693 } } }
I20260806 21:07:25.113809 6601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a8ad42c82ce4a948db32a55554092e4 [sys.catalog]: This master's current role is: LEADER
I20260806 21:07:25.117198 6608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:07:25.123039 6608 catalog_manager.cc:1347] Generated new cluster ID: 6dfeb36c8b4d45cf81ccd5ae5fc515aa
I20260806 21:07:25.123209 6608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:07:25.140635 6608 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:07:25.142055 6608 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:07:25.153383 6608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a8ad42c82ce4a948db32a55554092e4: Generated new TSK 0
I20260806 21:07:25.154327 6608 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.225.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
I20260806 21:07:25.193485 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40693
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:41819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:07:25.312345 6620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:07:25.312731 6620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:07:25.312826 6620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:07:25.317237 6620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:07:25.317353 6620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:07:25.317456 6620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:07:25.323339 6620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:41819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40693
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:07:25.324503 6620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:07:25.325543 6620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:07:25.336057 6626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:25.336028 6625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:25.336272 6628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:25.336365 6620 server_base.cc:1034] running on GCE node
I20260806 21:07:25.336813 6620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:07:25.337568 6620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:07:25.338763 6620 hybrid_clock.cc:648] HybridClock initialized: now 1786050445338765 us; error 50 us; skew 500 ppm
Aug 06 21:07:25 dist-test-slave-69rg krb5kdc[6513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050445, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:07:25.341780 6620 init.cc:377] Logged in from keytab as kudu/127.27.225.193@KRBTEST.COM (short username kudu)
I20260806 21:07:25.342787 6620 webserver.cc:460] Webserver started at http://127.27.225.193:41771/ using document root <none> and password file <none>
I20260806 21:07:25.343324 6620 fs_manager.cc:359] Metadata directory not provided
I20260806 21:07:25.343385 6620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:07:25.343550 6620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:07:25.345649 6620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "f5eb35ccbea34e68bdddba227130db59"
format_stamp: "Formatted at 2026-08-06 21:07:25 on dist-test-slave-69rg"
server_key: "0d17a23875419bba1b285f0d6d96d65d"
server_key_iv: "4e1467f3b6a381a9e460e67aeab31ccf"
server_key_version: "encryptionkey@0"
I20260806 21:07:25.346349 6620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "f5eb35ccbea34e68bdddba227130db59"
format_stamp: "Formatted at 2026-08-06 21:07:25 on dist-test-slave-69rg"
server_key: "0d17a23875419bba1b285f0d6d96d65d"
server_key_iv: "4e1467f3b6a381a9e460e67aeab31ccf"
server_key_version: "encryptionkey@0"
I20260806 21:07:25.350893 6620 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:07:25.354149 6636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:25.355505 6620 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:07:25.355628 6620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "f5eb35ccbea34e68bdddba227130db59"
format_stamp: "Formatted at 2026-08-06 21:07:25 on dist-test-slave-69rg"
server_key: "0d17a23875419bba1b285f0d6d96d65d"
server_key_iv: "4e1467f3b6a381a9e460e67aeab31ccf"
server_key_version: "encryptionkey@0"
I20260806 21:07:25.355783 6620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:07:25.373683 6620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:07:25.377265 6620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:07:25.377485 6620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:07:25.378155 6620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:07:25.379006 6620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:07:25.379055 6620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:25.379120 6620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:07:25.379153 6620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:25.388530 6620 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:41977
I20260806 21:07:25.388551 6750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:41977 every 8 connection(s)
I20260806 21:07:25.389472 6620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:07:25.391935 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 6620
I20260806 21:07:25.392076 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:07:25.392426 28551 external_mini_cluster.cc:1428] Setting key 273d88125f6bb1903102752747bcfc77
Aug 06 21:07:25 dist-test-slave-69rg krb5kdc[6513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050445, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.225.194@KRBTEST.COM; defaulting to no policy
I20260806 21:07:25.405450 6597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:25.391419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:39337 (local address 127.27.225.254:40693)
0806 21:07:25.391584 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:25.391589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:25.392496 (+ 907us) server_negotiation.cc:407] Connection header received
0806 21:07:25.394176 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:25.394180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:25.394235 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:25.394340 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:25.395521 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.396058 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.397214 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.397453 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.397570 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:25.400716 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:25.400724 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:25.400728 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:07:25.400767 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:25.402900 (+ 2133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:25.403567 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:25.403574 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:25.403579 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:25.403702 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:25.404118 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:25.404124 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:25.404127 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:25.404436 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:25.404559 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:25.405105 (+ 546us) server_negotiation.cc:299] Negotiation successful
0806 21:07:25.405240 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 21:07:25.406589 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:25.391794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40693 (local address 127.27.225.193:39337)
0806 21:07:25.392307 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:25.392361 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:25.393929 (+ 1568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:25.394516 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:25.394523 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:25.394893 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:25.395367 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:25.395381 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.396202 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.396205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:25.397078 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:25.397086 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.397611 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.397615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:25.397795 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:25.398526 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:25.398542 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:25.400497 (+ 1955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:07:25.403054 (+ 2557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:25.403061 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:25.403075 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:25.403424 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:25.403830 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:25.403832 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:25.403834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:25.404008 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:25.404575 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:07:25.404580 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:07:25.404863 (+ 283us) client_negotiation.cc:769] Sending connection context
0806 21:07:25.405042 (+ 179us) client_negotiation.cc:240] Negotiation successful
0806 21:07:25.405430 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":149,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.194@KRBTEST.COM" created.
I20260806 21:07:25.408187 6751 heartbeater.cc:347] Connected to a master server at 127.27.225.254:40693
I20260806 21:07:25.408452 6751 heartbeater.cc:464] Registering TS with master...
I20260806 21:07:25.408986 6751 heartbeater.cc:511] Master 127.27.225.254:40693 requested a full tablet report, sending...
I20260806 21:07:25.410786 6558 ts_manager.cc:194] Registered new tserver with Master: f5eb35ccbea34e68bdddba227130db59 (127.27.225.193:41977)
I20260806 21:07:25.412139 6558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.193@KRBTEST.COM'} at 127.27.225.193:39337
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
I20260806 21:07:25.430004 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40693
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:41819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:07:25.543306 6757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:07:25.543596 6757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:07:25.543640 6757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:07:25.547823 6757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:07:25.547890 6757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:07:25.547948 6757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:07:25.553330 6757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:41819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40693
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:07:25.554643 6757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:07:25.555732 6757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:07:25.567081 6762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:25.567054 6765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:25.567045 6763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:25.567251 6757 server_base.cc:1034] running on GCE node
I20260806 21:07:25.567803 6757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:07:25.568470 6757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:07:25.569671 6757 hybrid_clock.cc:648] HybridClock initialized: now 1786050445569658 us; error 30 us; skew 500 ppm
Aug 06 21:07:25 dist-test-slave-69rg krb5kdc[6513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050445, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:07:25.581601 6757 init.cc:377] Logged in from keytab as kudu/127.27.225.194@KRBTEST.COM (short username kudu)
I20260806 21:07:25.582653 6757 webserver.cc:460] Webserver started at http://127.27.225.194:39597/ using document root <none> and password file <none>
I20260806 21:07:25.583245 6757 fs_manager.cc:359] Metadata directory not provided
I20260806 21:07:25.583341 6757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:07:25.583582 6757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:07:25.585647 6757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "1de24f9c736043068cc42da2b621c6b0"
format_stamp: "Formatted at 2026-08-06 21:07:25 on dist-test-slave-69rg"
server_key: "a4101d069201dc273ba83c153200275e"
server_key_iv: "a747243f081a7cab200a005d1c54cb24"
server_key_version: "encryptionkey@0"
I20260806 21:07:25.586318 6757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "1de24f9c736043068cc42da2b621c6b0"
format_stamp: "Formatted at 2026-08-06 21:07:25 on dist-test-slave-69rg"
server_key: "a4101d069201dc273ba83c153200275e"
server_key_iv: "a747243f081a7cab200a005d1c54cb24"
server_key_version: "encryptionkey@0"
I20260806 21:07:25.590179 6757 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 21:07:25.592700 6772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:25.594100 6757 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:07:25.594255 6757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "1de24f9c736043068cc42da2b621c6b0"
format_stamp: "Formatted at 2026-08-06 21:07:25 on dist-test-slave-69rg"
server_key: "a4101d069201dc273ba83c153200275e"
server_key_iv: "a747243f081a7cab200a005d1c54cb24"
server_key_version: "encryptionkey@0"
I20260806 21:07:25.594390 6757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:07:25.612059 6757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:07:25.616441 6757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:07:25.616622 6757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:07:25.617187 6757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:07:25.618129 6757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:07:25.618184 6757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:25.618228 6757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:07:25.618251 6757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:25.627501 6757 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:37953
I20260806 21:07:25.627573 6886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:37953 every 8 connection(s)
I20260806 21:07:25.628549 6757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:07:25.638120 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 6757
I20260806 21:07:25.638300 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:07:25.638623 28551 external_mini_cluster.cc:1428] Setting key 8e3a372cb82bf60d1182163f182a0d74
Aug 06 21:07:25 dist-test-slave-69rg krb5kdc[6513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050445, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:07:25.644122 6597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:25.630733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:50897 (local address 127.27.225.254:40693)
0806 21:07:25.630922 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:25.630926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:25.631822 (+ 896us) server_negotiation.cc:407] Connection header received
0806 21:07:25.633278 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:25.633281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:25.633343 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:25.633458 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:25.634789 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.635367 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.636556 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.636789 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.636903 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:25.639746 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:25.639750 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:25.639753 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:07:25.639784 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:25.641659 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:25.642344 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:25.642351 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:25.642356 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:25.642453 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:25.642906 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:25.642908 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:25.642910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:25.643094 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:25.643176 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:25.643867 (+ 691us) server_negotiation.cc:299] Negotiation successful
0806 21:07:25.643993 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 21:07:25.645377 6890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:25.631101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40693 (local address 127.27.225.194:50897)
0806 21:07:25.631621 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:25.631690 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:25.633049 (+ 1359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:25.633644 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:25.633651 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:25.634129 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:25.634624 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:25.634647 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.635507 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.635515 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:25.636419 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:25.636429 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.636959 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.636963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:25.637140 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:25.637753 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:25.637770 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:25.639570 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:07:25.641785 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:25.641789 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:25.641801 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:25.642234 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:25.642564 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:25.642569 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:25.642573 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:25.642794 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:25.643200 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:07:25.643206 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:07:25.643554 (+ 348us) client_negotiation.cc:769] Sending connection context
0806 21:07:25.643795 (+ 241us) client_negotiation.cc:240] Negotiation successful
0806 21:07:25.644191 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":120,"threads_started":1}
I20260806 21:07:25.646778 6888 heartbeater.cc:347] Connected to a master server at 127.27.225.254:40693
I20260806 21:07:25.647037 6888 heartbeater.cc:464] Registering TS with master...
I20260806 21:07:25.647558 6888 heartbeater.cc:511] Master 127.27.225.254:40693 requested a full tablet report, sending...
I20260806 21:07:25.648644 6558 ts_manager.cc:194] Registered new tserver with Master: 1de24f9c736043068cc42da2b621c6b0 (127.27.225.194:37953)
I20260806 21:07:25.649293 6558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.194@KRBTEST.COM'} at 127.27.225.194:50897
WARNING: no policy specified for kudu/127.27.225.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
I20260806 21:07:25.674171 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40693
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:41819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:07:25.788702 6893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:07:25.789026 6893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:07:25.789114 6893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:07:25.793166 6893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:07:25.793280 6893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:07:25.793378 6893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:07:25.798928 6893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:41819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40693
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:07:25.800112 6893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:07:25.801362 6893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:07:25.812203 6899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:25.812259 6901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:07:25.812249 6898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:07:25.812368 6893 server_base.cc:1034] running on GCE node
I20260806 21:07:25.813043 6893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:07:25.813735 6893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:07:25.814985 6893 hybrid_clock.cc:648] HybridClock initialized: now 1786050445814936 us; error 60 us; skew 500 ppm
Aug 06 21:07:25 dist-test-slave-69rg krb5kdc[6513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050445, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:07:25.818287 6893 init.cc:377] Logged in from keytab as kudu/127.27.225.195@KRBTEST.COM (short username kudu)
I20260806 21:07:25.819283 6893 webserver.cc:460] Webserver started at http://127.27.225.195:36403/ using document root <none> and password file <none>
I20260806 21:07:25.819931 6893 fs_manager.cc:359] Metadata directory not provided
I20260806 21:07:25.820042 6893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:07:25.820302 6893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:07:25.822655 6893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "f602cdc88f1047c8a95db0a05f3b5a81"
format_stamp: "Formatted at 2026-08-06 21:07:25 on dist-test-slave-69rg"
server_key: "5e3858fe127cffeb9bac3ec199063af4"
server_key_iv: "5adcccfba0e285c3b1c96480a6e04469"
server_key_version: "encryptionkey@0"
I20260806 21:07:25.823272 6893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "f602cdc88f1047c8a95db0a05f3b5a81"
format_stamp: "Formatted at 2026-08-06 21:07:25 on dist-test-slave-69rg"
server_key: "5e3858fe127cffeb9bac3ec199063af4"
server_key_iv: "5adcccfba0e285c3b1c96480a6e04469"
server_key_version: "encryptionkey@0"
I20260806 21:07:25.827625 6893 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:07:25.830250 6908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:25.831493 6893 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:07:25.831643 6893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "f602cdc88f1047c8a95db0a05f3b5a81"
format_stamp: "Formatted at 2026-08-06 21:07:25 on dist-test-slave-69rg"
server_key: "5e3858fe127cffeb9bac3ec199063af4"
server_key_iv: "5adcccfba0e285c3b1c96480a6e04469"
server_key_version: "encryptionkey@0"
I20260806 21:07:25.831766 6893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:07:25.851354 6893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:07:25.854874 6893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:07:25.855090 6893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:07:25.855721 6893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:07:25.856590 6893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:07:25.856637 6893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:25.856704 6893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:07:25.856736 6893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:07:25.866357 6893 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:32879
I20260806 21:07:25.866402 7026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:32879 every 8 connection(s)
I20260806 21:07:25.867293 6893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:07:25.872360 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 6893
I20260806 21:07:25.872490 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:07:25.872833 28551 external_mini_cluster.cc:1428] Setting key 741272d43856d5c1b18614ebb32c10de
Aug 06 21:07:25 dist-test-slave-69rg krb5kdc[6513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050445, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:07:25.882766 6597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:25.869353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:45131 (local address 127.27.225.254:40693)
0806 21:07:25.869545 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:25.869551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:25.870466 (+ 915us) server_negotiation.cc:407] Connection header received
0806 21:07:25.871986 (+ 1520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:25.871990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:25.872092 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:25.872206 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:25.873678 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.874523 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.876072 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.876294 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.876384 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:25.878888 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:25.878892 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:25.878894 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:07:25.878920 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:25.880539 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:25.881093 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:25.881095 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:25.881097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:25.881218 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:25.881586 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:25.881589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:25.881590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:25.881788 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:25.881962 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:25.882522 (+ 560us) server_negotiation.cc:299] Negotiation successful
0806 21:07:25.882639 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 21:07:25.883632 7029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:25.869747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40693 (local address 127.27.225.195:45131)
0806 21:07:25.870282 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:25.870330 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:25.871763 (+ 1433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:25.872405 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:25.872412 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:25.872939 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:07:25.873493 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:25.873507 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.874680 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.874684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:25.875872 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:25.875883 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.876434 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.876436 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:25.876575 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:25.877218 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:07:25.877233 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:07:25.878729 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:07:25.880674 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:25.880680 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:25.880692 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:25.880983 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:25.881327 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:07:25.881329 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:07:25.881331 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:07:25.881482 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:07:25.881963 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:07:25.881968 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:07:25.882260 (+ 292us) client_negotiation.cc:769] Sending connection context
0806 21:07:25.882437 (+ 177us) client_negotiation.cc:240] Negotiation successful
0806 21:07:25.882713 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":144,"threads_started":1}
I20260806 21:07:25.884918 7027 heartbeater.cc:347] Connected to a master server at 127.27.225.254:40693
I20260806 21:07:25.885219 7027 heartbeater.cc:464] Registering TS with master...
I20260806 21:07:25.885717 7027 heartbeater.cc:511] Master 127.27.225.254:40693 requested a full tablet report, sending...
I20260806 21:07:25.886790 6558 ts_manager.cc:194] Registered new tserver with Master: f602cdc88f1047c8a95db0a05f3b5a81 (127.27.225.195:32879)
I20260806 21:07:25.887435 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:07:25.887457 6558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.195@KRBTEST.COM'} at 127.27.225.195:45131
I20260806 21:07:25.898619 6597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:25.889772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53180 (local address 127.27.225.254:40693)
0806 21:07:25.889955 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:25.889960 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:25.890175 (+ 215us) server_negotiation.cc:407] Connection header received
0806 21:07:25.890282 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:25.890284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:25.890352 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:25.890458 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:07:25.891273 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.891911 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.893022 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.893271 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.893363 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:25.894873 (+ 1510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:07:25.894876 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:07:25.894878 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:07:25.894902 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:07:25.896443 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:25.896939 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:25.896942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:25.896943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:25.897035 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:07:25.897568 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:07:25.897572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:07:25.897573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:07:25.897888 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:07:25.898028 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:25.898352 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 21:07:25.898488 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260806 21:07:25.902135 6558 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:07:25.905289 6558 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:07:25.921178 7038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:25.914926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:32879 (local address 127.0.0.1:39142)
0806 21:07:25.915654 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:25.915723 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:25.915858 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:25.916710 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:25.916716 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:25.916740 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:07:25.916901 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:25.916908 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.918375 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.918378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:25.919829 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:25.919840 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.920734 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.920737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:25.920823 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:25.920857 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 21:07:25.920901 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 21:07:25.921021 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"mutex_wait_us":4,"thread_start_us":94,"threads_started":1}
I20260806 21:07:25.921239 7042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:25.915743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59360 (local address 127.27.225.193:41977)
0806 21:07:25.916492 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:25.916500 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:25.916523 (+ 23us) server_negotiation.cc:407] Connection header received
0806 21:07:25.916608 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:25.916612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:25.916783 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:25.916925 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:07:25.917390 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.918511 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.919633 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.920409 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.920552 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:25.920693 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:25.920994 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 21:07:25.921091 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":85,"threads_started":1}
I20260806 21:07:25.921348 7041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:25.915722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:37953 (local address 127.0.0.1:50306)
0806 21:07:25.916163 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:25.916174 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:25.916239 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:25.917075 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:25.917078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:25.917102 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:07:25.917267 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:25.917273 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.918695 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.918699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:25.919841 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:25.919848 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.920914 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.920918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:25.921014 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:25.921036 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 21:07:25.921116 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 21:07:25.921203 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":60,"thread_start_us":106,"threads_started":1}
I20260806 21:07:25.921942 7043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:25.916152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50306 (local address 127.27.225.194:37953)
0806 21:07:25.916492 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:25.916502 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:25.916532 (+ 30us) server_negotiation.cc:407] Connection header received
0806 21:07:25.916602 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:25.916606 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:25.916783 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:25.916924 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:07:25.917390 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.918504 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.919979 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.920784 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.920917 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:25.921641 (+ 724us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:25.921685 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 21:07:25.921755 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":128,"threads_started":1}
I20260806 21:07:25.922065 7040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:25.915253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:41977 (local address 127.0.0.1:59360)
0806 21:07:25.915832 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:25.915848 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:25.915940 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:25.917075 (+ 1135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:25.917078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:25.917096 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:07:25.917239 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:25.917249 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.918638 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.918643 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:25.919490 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:25.919500 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.920570 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.920574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:25.920711 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:25.920760 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 21:07:25.920869 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 21:07:25.921938 (+ 1069us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"mutex_wait_us":957,"thread_start_us":111,"threads_started":1}
I20260806 21:07:25.922781 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:25.915045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39142 (local address 127.27.225.195:32879)
0806 21:07:25.916201 (+ 1156us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:25.916213 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:25.916238 (+ 25us) server_negotiation.cc:407] Connection header received
0806 21:07:25.916325 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:25.916331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:25.916543 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:25.916728 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:07:25.917025 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.918221 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.919998 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.920589 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.922379 (+ 1790us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:25.922498 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:25.922543 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 21:07:25.922628 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1049,"thread_start_us":205,"threads_started":1}
I20260806 21:07:25.924948 6821 tablet_service.cc:1467] Processing CreateTablet for tablet 513fd9bd3afc45349b9874a0114adbee (DEFAULT_TABLE table=test-table [id=d22762bc8dd449939d1ec1b6f580fed6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:07:25.924947 6685 tablet_service.cc:1467] Processing CreateTablet for tablet 513fd9bd3afc45349b9874a0114adbee (DEFAULT_TABLE table=test-table [id=d22762bc8dd449939d1ec1b6f580fed6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:07:25.926127 6685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 513fd9bd3afc45349b9874a0114adbee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:07:25.926127 6821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 513fd9bd3afc45349b9874a0114adbee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:07:25.927517 6958 tablet_service.cc:1467] Processing CreateTablet for tablet 513fd9bd3afc45349b9874a0114adbee (DEFAULT_TABLE table=test-table [id=d22762bc8dd449939d1ec1b6f580fed6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:07:25.928345 6958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 513fd9bd3afc45349b9874a0114adbee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:07:25.933485 7044 tablet_bootstrap.cc:492] T 513fd9bd3afc45349b9874a0114adbee P f5eb35ccbea34e68bdddba227130db59: Bootstrap starting.
I20260806 21:07:25.934803 7046 tablet_bootstrap.cc:492] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81: Bootstrap starting.
I20260806 21:07:25.934803 7045 tablet_bootstrap.cc:492] T 513fd9bd3afc45349b9874a0114adbee P 1de24f9c736043068cc42da2b621c6b0: Bootstrap starting.
I20260806 21:07:25.936709 7044 tablet_bootstrap.cc:654] T 513fd9bd3afc45349b9874a0114adbee P f5eb35ccbea34e68bdddba227130db59: Neither blocks nor log segments found. Creating new log.
I20260806 21:07:25.937394 7046 tablet_bootstrap.cc:654] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81: Neither blocks nor log segments found. Creating new log.
I20260806 21:07:25.937441 7045 tablet_bootstrap.cc:654] T 513fd9bd3afc45349b9874a0114adbee P 1de24f9c736043068cc42da2b621c6b0: Neither blocks nor log segments found. Creating new log.
I20260806 21:07:25.937441 7044 log.cc:824] T 513fd9bd3afc45349b9874a0114adbee P f5eb35ccbea34e68bdddba227130db59: Log is configured to *not* fsync() on all Append() calls
I20260806 21:07:25.938294 7045 log.cc:824] T 513fd9bd3afc45349b9874a0114adbee P 1de24f9c736043068cc42da2b621c6b0: Log is configured to *not* fsync() on all Append() calls
I20260806 21:07:25.938294 7046 log.cc:824] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81: Log is configured to *not* fsync() on all Append() calls
I20260806 21:07:25.939602 7044 tablet_bootstrap.cc:492] T 513fd9bd3afc45349b9874a0114adbee P f5eb35ccbea34e68bdddba227130db59: No bootstrap required, opened a new log
I20260806 21:07:25.939822 7044 ts_tablet_manager.cc:1397] T 513fd9bd3afc45349b9874a0114adbee P f5eb35ccbea34e68bdddba227130db59: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 21:07:25.940217 7045 tablet_bootstrap.cc:492] T 513fd9bd3afc45349b9874a0114adbee P 1de24f9c736043068cc42da2b621c6b0: No bootstrap required, opened a new log
I20260806 21:07:25.940403 7045 ts_tablet_manager.cc:1397] T 513fd9bd3afc45349b9874a0114adbee P 1de24f9c736043068cc42da2b621c6b0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:07:25.940618 7046 tablet_bootstrap.cc:492] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81: No bootstrap required, opened a new log
I20260806 21:07:25.940788 7046 ts_tablet_manager.cc:1397] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:07:25.944108 7044 raft_consensus.cc:348] T 513fd9bd3afc45349b9874a0114adbee P f5eb35ccbea34e68bdddba227130db59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f602cdc88f1047c8a95db0a05f3b5a81" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 32879 } } peers { permanent_uuid: "1de24f9c736043068cc42da2b621c6b0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37953 } } peers { permanent_uuid: "f5eb35ccbea34e68bdddba227130db59" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 41977 } }
I20260806 21:07:25.944375 7044 raft_consensus.cc:374] T 513fd9bd3afc45349b9874a0114adbee P f5eb35ccbea34e68bdddba227130db59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:07:25.944430 7044 raft_consensus.cc:729] T 513fd9bd3afc45349b9874a0114adbee P f5eb35ccbea34e68bdddba227130db59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5eb35ccbea34e68bdddba227130db59, State: Initialized, Role: FOLLOWER
I20260806 21:07:25.944293 7045 raft_consensus.cc:348] T 513fd9bd3afc45349b9874a0114adbee P 1de24f9c736043068cc42da2b621c6b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f602cdc88f1047c8a95db0a05f3b5a81" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 32879 } } peers { permanent_uuid: "1de24f9c736043068cc42da2b621c6b0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37953 } } peers { permanent_uuid: "f5eb35ccbea34e68bdddba227130db59" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 41977 } }
I20260806 21:07:25.944557 7045 raft_consensus.cc:374] T 513fd9bd3afc45349b9874a0114adbee P 1de24f9c736043068cc42da2b621c6b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:07:25.944625 7045 raft_consensus.cc:729] T 513fd9bd3afc45349b9874a0114adbee P 1de24f9c736043068cc42da2b621c6b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1de24f9c736043068cc42da2b621c6b0, State: Initialized, Role: FOLLOWER
I20260806 21:07:25.944440 7046 raft_consensus.cc:348] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f602cdc88f1047c8a95db0a05f3b5a81" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 32879 } } peers { permanent_uuid: "1de24f9c736043068cc42da2b621c6b0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37953 } } peers { permanent_uuid: "f5eb35ccbea34e68bdddba227130db59" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 41977 } }
I20260806 21:07:25.944696 7046 raft_consensus.cc:374] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:07:25.944759 7046 raft_consensus.cc:729] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f602cdc88f1047c8a95db0a05f3b5a81, State: Initialized, Role: FOLLOWER
I20260806 21:07:25.944976 7044 consensus_queue.cc:260] T 513fd9bd3afc45349b9874a0114adbee P f5eb35ccbea34e68bdddba227130db59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f602cdc88f1047c8a95db0a05f3b5a81" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 32879 } } peers { permanent_uuid: "1de24f9c736043068cc42da2b621c6b0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37953 } } peers { permanent_uuid: "f5eb35ccbea34e68bdddba227130db59" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 41977 } }
I20260806 21:07:25.945104 7045 consensus_queue.cc:260] T 513fd9bd3afc45349b9874a0114adbee P 1de24f9c736043068cc42da2b621c6b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f602cdc88f1047c8a95db0a05f3b5a81" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 32879 } } peers { permanent_uuid: "1de24f9c736043068cc42da2b621c6b0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37953 } } peers { permanent_uuid: "f5eb35ccbea34e68bdddba227130db59" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 41977 } }
I20260806 21:07:25.945101 7046 consensus_queue.cc:260] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f602cdc88f1047c8a95db0a05f3b5a81" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 32879 } } peers { permanent_uuid: "1de24f9c736043068cc42da2b621c6b0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37953 } } peers { permanent_uuid: "f5eb35ccbea34e68bdddba227130db59" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 41977 } }
I20260806 21:07:25.945634 6751 heartbeater.cc:503] Master 127.27.225.254:40693 was elected leader, sending a full tablet report...
I20260806 21:07:25.946130 7046 ts_tablet_manager.cc:1428] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 21:07:25.946130 7045 ts_tablet_manager.cc:1428] T 513fd9bd3afc45349b9874a0114adbee P 1de24f9c736043068cc42da2b621c6b0: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 21:07:25.946485 7027 heartbeater.cc:503] Master 127.27.225.254:40693 was elected leader, sending a full tablet report...
I20260806 21:07:25.947084 7044 ts_tablet_manager.cc:1428] T 513fd9bd3afc45349b9874a0114adbee P f5eb35ccbea34e68bdddba227130db59: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260806 21:07:25.947428 6888 heartbeater.cc:503] Master 127.27.225.254:40693 was elected leader, sending a full tablet report...
I20260806 21:07:25.948606 7052 raft_consensus.cc:482] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:07:25.948750 7052 raft_consensus.cc:504] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f602cdc88f1047c8a95db0a05f3b5a81" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 32879 } } peers { permanent_uuid: "1de24f9c736043068cc42da2b621c6b0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37953 } } peers { permanent_uuid: "f5eb35ccbea34e68bdddba227130db59" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 41977 } }
I20260806 21:07:25.950022 7052 leader_election.cc:288] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1de24f9c736043068cc42da2b621c6b0 (127.27.225.194:37953), f5eb35ccbea34e68bdddba227130db59 (127.27.225.193:41977)
I20260806 21:07:25.954106 7029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:25.950195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:37953 (local address 127.27.225.195:57051)
0806 21:07:25.950452 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:25.950470 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:25.950545 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:25.950924 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:25.950930 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:25.950949 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:07:25.951089 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:25.951096 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.952091 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.952094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:25.952890 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:25.952899 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.953652 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.953655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:25.953766 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:25.953792 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 21:07:25.953896 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 21:07:25.953973 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"spinlock_wait_cycles":7296}
I20260806 21:07:25.954178 7043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:25.950477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:57051 (local address 127.27.225.194:37953)
0806 21:07:25.950604 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:25.950608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:25.950620 (+ 12us) server_negotiation.cc:407] Connection header received
0806 21:07:25.950659 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:25.950661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:25.950695 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:25.950803 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:07:25.951210 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.951970 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.953024 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.953537 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.953630 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:25.953668 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:25.953913 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 21:07:25.953988 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 21:07:25.954370 7054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:25.950550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:41977 (local address 127.27.225.195:50931)
0806 21:07:25.950883 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:25.950900 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:07:25.950979 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:07:25.951277 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:07:25.951280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:07:25.951297 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:07:25.951395 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:25.951401 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.952321 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.952323 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:25.953176 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:07:25.953186 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.953955 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.953959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:07:25.954050 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:25.954072 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 21:07:25.954147 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 21:07:25.954219 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"spinlock_wait_cycles":384,"thread_start_us":129,"threads_started":1}
I20260806 21:07:25.954403 7042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:25.950942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:50931 (local address 127.27.225.193:41977)
0806 21:07:25.951052 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:25.951058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:25.951074 (+ 16us) server_negotiation.cc:407] Connection header received
0806 21:07:25.951119 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:25.951122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:25.951193 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:25.951274 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:07:25.951525 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.952205 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.953305 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.953842 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.953945 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:25.953988 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:25.954185 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 21:07:25.954267 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 21:07:25.954967 6705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "513fd9bd3afc45349b9874a0114adbee" candidate_uuid: "f602cdc88f1047c8a95db0a05f3b5a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5eb35ccbea34e68bdddba227130db59" is_pre_election: true
I20260806 21:07:25.955096 6841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "513fd9bd3afc45349b9874a0114adbee" candidate_uuid: "f602cdc88f1047c8a95db0a05f3b5a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1de24f9c736043068cc42da2b621c6b0" is_pre_election: true
I20260806 21:07:25.955258 6705 raft_consensus.cc:2454] T 513fd9bd3afc45349b9874a0114adbee P f5eb35ccbea34e68bdddba227130db59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f602cdc88f1047c8a95db0a05f3b5a81 in term 0.
I20260806 21:07:25.955435 6841 raft_consensus.cc:2454] T 513fd9bd3afc45349b9874a0114adbee P 1de24f9c736043068cc42da2b621c6b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f602cdc88f1047c8a95db0a05f3b5a81 in term 0.
I20260806 21:07:25.955729 6912 leader_election.cc:302] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f5eb35ccbea34e68bdddba227130db59, f602cdc88f1047c8a95db0a05f3b5a81; no voters:
I20260806 21:07:25.955971 7052 raft_consensus.cc:2781] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:07:25.956053 7052 raft_consensus.cc:482] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:07:25.956103 7052 raft_consensus.cc:3037] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:07:25.957103 7052 raft_consensus.cc:504] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f602cdc88f1047c8a95db0a05f3b5a81" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 32879 } } peers { permanent_uuid: "1de24f9c736043068cc42da2b621c6b0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37953 } } peers { permanent_uuid: "f5eb35ccbea34e68bdddba227130db59" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 41977 } }
I20260806 21:07:25.957419 7052 leader_election.cc:288] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81 [CANDIDATE]: Term 1 election: Requested vote from peers 1de24f9c736043068cc42da2b621c6b0 (127.27.225.194:37953), f5eb35ccbea34e68bdddba227130db59 (127.27.225.193:41977)
I20260806 21:07:25.957696 6841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "513fd9bd3afc45349b9874a0114adbee" candidate_uuid: "f602cdc88f1047c8a95db0a05f3b5a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1de24f9c736043068cc42da2b621c6b0"
I20260806 21:07:25.957764 6705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "513fd9bd3afc45349b9874a0114adbee" candidate_uuid: "f602cdc88f1047c8a95db0a05f3b5a81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5eb35ccbea34e68bdddba227130db59"
I20260806 21:07:25.957856 6841 raft_consensus.cc:3037] T 513fd9bd3afc45349b9874a0114adbee P 1de24f9c736043068cc42da2b621c6b0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:07:25.957922 6705 raft_consensus.cc:3037] T 513fd9bd3afc45349b9874a0114adbee P f5eb35ccbea34e68bdddba227130db59 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:07:25.958985 6705 raft_consensus.cc:2454] T 513fd9bd3afc45349b9874a0114adbee P f5eb35ccbea34e68bdddba227130db59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f602cdc88f1047c8a95db0a05f3b5a81 in term 1.
I20260806 21:07:25.959000 6841 raft_consensus.cc:2454] T 513fd9bd3afc45349b9874a0114adbee P 1de24f9c736043068cc42da2b621c6b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f602cdc88f1047c8a95db0a05f3b5a81 in term 1.
I20260806 21:07:25.959261 6912 leader_election.cc:302] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f5eb35ccbea34e68bdddba227130db59, f602cdc88f1047c8a95db0a05f3b5a81; no voters:
I20260806 21:07:25.959438 7052 raft_consensus.cc:2781] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:07:25.959666 7052 raft_consensus.cc:686] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81 [term 1 LEADER]: Becoming Leader. State: Replica: f602cdc88f1047c8a95db0a05f3b5a81, State: Running, Role: LEADER
I20260806 21:07:25.960013 7052 consensus_queue.cc:237] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f602cdc88f1047c8a95db0a05f3b5a81" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 32879 } } peers { permanent_uuid: "1de24f9c736043068cc42da2b621c6b0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37953 } } peers { permanent_uuid: "f5eb35ccbea34e68bdddba227130db59" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 41977 } }
I20260806 21:07:25.963255 6555 catalog_manager.cc:5495] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81 reported cstate change: term changed from 0 to 1, leader changed from <none> to f602cdc88f1047c8a95db0a05f3b5a81 (127.27.225.195). New cstate: current_term: 1 leader_uuid: "f602cdc88f1047c8a95db0a05f3b5a81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f602cdc88f1047c8a95db0a05f3b5a81" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 32879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1de24f9c736043068cc42da2b621c6b0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5eb35ccbea34e68bdddba227130db59" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 41977 } health_report { overall_health: UNKNOWN } } }
I20260806 21:07:25.983198 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:25.979717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39158 (local address 127.27.225.195:32879)
0806 21:07:25.979856 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 21:07:25.979860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:07:25.979873 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:07:25.979946 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:07:25.979948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:07:25.980007 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:07:25.980098 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:07:25.980343 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.980857 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.981728 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:07:25.982010 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:07:25.982140 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:07:25.982360 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:07:25.982779 (+ 419us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:07:25.982866 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 21:07:25.982976 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 21:07:25.983031 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 21:07:25.988974 6705 raft_consensus.cc:1261] T 513fd9bd3afc45349b9874a0114adbee P f5eb35ccbea34e68bdddba227130db59 [term 1 FOLLOWER]: Refusing update from remote peer f602cdc88f1047c8a95db0a05f3b5a81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:07:25.988974 6841 raft_consensus.cc:1261] T 513fd9bd3afc45349b9874a0114adbee P 1de24f9c736043068cc42da2b621c6b0 [term 1 FOLLOWER]: Refusing update from remote peer f602cdc88f1047c8a95db0a05f3b5a81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:07:25.989781 7055 consensus_queue.cc:1035] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1de24f9c736043068cc42da2b621c6b0" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:07:25.990062 7052 consensus_queue.cc:1035] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5eb35ccbea34e68bdddba227130db59" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 41977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:07:25.997529 7059 mvcc.cc:205] Tried to move back new op lower bound from 7315662626763554816 to 7315662626654932992. Current Snapshot: MvccSnapshot[applied={T|T < 7315662626763554816}]
I20260806 21:07:26.001703 7061 mvcc.cc:205] Tried to move back new op lower bound from 7315662626763554816 to 7315662626654932992. Current Snapshot: MvccSnapshot[applied={T|T < 7315662626763554816}]
I20260806 21:07:26.007295 6557 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:07:26.007520 6557 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:07:26.010290 6557 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5a8ad42c82ce4a948db32a55554092e4: Sending DeleteTablet for 3 replicas of tablet 513fd9bd3afc45349b9874a0114adbee
I20260806 21:07:26.011267 6821 tablet_service.cc:1514] Processing DeleteTablet for tablet 513fd9bd3afc45349b9874a0114adbee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:07:26 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:50306
I20260806 21:07:26.011267 6685 tablet_service.cc:1514] Processing DeleteTablet for tablet 513fd9bd3afc45349b9874a0114adbee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:07:26 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:59360
I20260806 21:07:26.011847 7068 tablet_replica.cc:330] T 513fd9bd3afc45349b9874a0114adbee P 1de24f9c736043068cc42da2b621c6b0: stopping tablet replica
I20260806 21:07:26.012007 6958 tablet_service.cc:1514] Processing DeleteTablet for tablet 513fd9bd3afc45349b9874a0114adbee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:07:26 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:39142
I20260806 21:07:26.012247 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 6620
I20260806 21:07:26.012329 7068 raft_consensus.cc:2229] T 513fd9bd3afc45349b9874a0114adbee P 1de24f9c736043068cc42da2b621c6b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:07:26.012722 7068 raft_consensus.cc:2258] T 513fd9bd3afc45349b9874a0114adbee P 1de24f9c736043068cc42da2b621c6b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:07:26.014681 7068 ts_tablet_manager.cc:1905] T 513fd9bd3afc45349b9874a0114adbee P 1de24f9c736043068cc42da2b621c6b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:07:26.017238 7068 ts_tablet_manager.cc:1918] T 513fd9bd3afc45349b9874a0114adbee P 1de24f9c736043068cc42da2b621c6b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:07:26.017401 7068 log.cc:1196] T 513fd9bd3afc45349b9874a0114adbee P 1de24f9c736043068cc42da2b621c6b0: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-1/wal/wals/513fd9bd3afc45349b9874a0114adbee
I20260806 21:07:26.017750 7068 ts_tablet_manager.cc:1939] T 513fd9bd3afc45349b9874a0114adbee P 1de24f9c736043068cc42da2b621c6b0: Deleting consensus metadata
I20260806 21:07:26.019157 6545 catalog_manager.cc:4841] TS 1de24f9c736043068cc42da2b621c6b0 (127.27.225.194:37953): tablet 513fd9bd3afc45349b9874a0114adbee (table test-table [id=d22762bc8dd449939d1ec1b6f580fed6]) successfully deleted
I20260806 21:07:26.022132 7070 tablet_replica.cc:330] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81: stopping tablet replica
I20260806 21:07:26.022521 7070 raft_consensus.cc:2229] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:07:26.022922 7070 raft_consensus.cc:2258] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 21:07:26.023460 6544 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.225.193:41977 (error 108)
I20260806 21:07:26.024408 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 6757
I20260806 21:07:26.026399 7070 ts_tablet_manager.cc:1905] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:07:26.030092 7070 ts_tablet_manager.cc:1918] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:07:26.030247 7070 log.cc:1196] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/513fd9bd3afc45349b9874a0114adbee
I20260806 21:07:26.030673 7070 ts_tablet_manager.cc:1939] T 513fd9bd3afc45349b9874a0114adbee P f602cdc88f1047c8a95db0a05f3b5a81: Deleting consensus metadata
I20260806 21:07:26.032045 6545 catalog_manager.cc:4841] TS f602cdc88f1047c8a95db0a05f3b5a81 (127.27.225.195:32879): tablet 513fd9bd3afc45349b9874a0114adbee (table test-table [id=d22762bc8dd449939d1ec1b6f580fed6]) successfully deleted
I20260806 21:07:26.034063 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 6893
I20260806 21:07:26.034266 7040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:07:26.033890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:41977 (local address 127.0.0.1:59372)
0806 21:07:26.034030 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 21:07:26.034117 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.193:41977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 21:07:26.036295 6544 catalog_manager.cc:4584] TS f5eb35ccbea34e68bdddba227130db59 (127.27.225.193:41977): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 513fd9bd3afc45349b9874a0114adbee: Network error: Client connection negotiation failed: client connection to 127.27.225.193:41977: connect: Connection refused (error 111)
I20260806 21:07:26.040984 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 6526
2026-08-06T21:07:26Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2378 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:07:26 dist-test-slave-69rg krb5kdc[7074](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:07:26 dist-test-slave-69rg krb5kdc[7074](info): set up 2 sockets
Aug 06 21:07:26 dist-test-slave-69rg krb5kdc[7074](info): commencing operation
krb5kdc: starting...
W20260806 21:07:27.111004 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:07:27 dist-test-slave-69rg krb5kdc[7074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:07:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:07:27Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.27.225.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.27.225.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.27.225.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/rangeradmin_127.27.225.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.27.225.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/rangeradmin_127.27.225.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.27.225.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.27.225.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.27.225.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/rangerlookup_127.27.225.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.27.225.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/rangerlookup_127.27.225.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.27.225.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.27.225.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.27.225.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/HTTP_127.27.225.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.27.225.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/HTTP_127.27.225.212@KRBTEST.COM.keytab.
I20260806 21:07:27.323649 28551 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDiceBW/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/postgres -l logfile start
2026-08-06 21:07:30.234 UTC [7104] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 21:07:30.234 UTC [7104] LOG: listening on IPv4 address "127.27.225.212", port 56623
2026-08-06 21:07:30.238 UTC [7104] LOG: listening on Unix socket "/tmp/.s.PGSQL.56623"
2026-08-06 21:07:30.275 UTC [7107] LOG: database system was shut down at 2026-08-06 21:07:28 UTC
2026-08-06 21:07:30.280 UTC [7104] LOG: database system is ready to accept connections
I20260806 21:07:31.268260 28551 mini_postgres.cc:96] Postgres bound to 56623
2026-08-06 21:07:31.276 UTC [7115] FATAL: database "slave" does not exist
127.27.225.212:56623 - accepting connections
I20260806 21:07:31.277627 28551 mini_ranger.cc:161] Starting Ranger...
I20260806 21:07:31.297475 28551 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 21:07:31.894007 28551 mini_ranger.cc:87] Created ranger Postgres database
I20260806 21:07:31.894111 28551 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ranger-admin
2026-08-06 21:07:32,188 [I] DB FLAVOR :POSTGRES
2026-08-06 21:07:32,188 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 21:07:32,188 [I] Checking connection..
2026-08-06 21:07:32,189 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:56623/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 21:07:32,457 [I] Checking connection passed.
2026-08-06 21:07:32,458 [I] --------- Verifying version history table ---------
2026-08-06 21:07:32,458 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:56623/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 21:07:32,710 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 21:07:32,711 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:56623/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 21:07:32,966 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 21:07:32,966 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 21:07:32,966 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:56623/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 21:07:33.217 UTC [7181] WARNING: there is no transaction in progress
2026-08-06 21:07:33,225 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 21:07:33,226 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:56623/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 21:07:33,486 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 21:07:33,486 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 21:07:33,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:56623/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 21:07:33,738 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:56623/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 21:07:33,987 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:56623/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-69rg.c.gcp-upstream.internal','N') ;"
2026-08-06 21:07:34,240 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:56623/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 21:07:34,506 [I] Table x_portal_user does not exist in database ranger
2026-08-06 21:07:34,507 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:56623/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 21:07:34,778 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 21:07:34,778 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:56623/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 21:07:35,053 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:56623/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 21:07:35,330 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 21:07:35,330 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:56623/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 21:07:36.470 UTC [7291] WARNING: there is no transaction in progress
2026-08-06 21:07:36.491 UTC [7291] WARNING: there is no transaction in progress
2026-08-06 21:07:36.510 UTC [7291] WARNING: there is no transaction in progress
2026-08-06 21:07:36.529 UTC [7291] WARNING: there is no transaction in progress
2026-08-06 21:07:36.549 UTC [7291] WARNING: there is no transaction in progress
2026-08-06 21:07:36.569 UTC [7291] WARNING: there is no transaction in progress
2026-08-06 21:07:36.637 UTC [7291] WARNING: there is no transaction in progress
2026-08-06 21:07:36.646 UTC [7291] WARNING: there is no transaction in progress
2026-08-06 21:07:36.665 UTC [7291] WARNING: there is no transaction in progress
2026-08-06 21:07:36.679 UTC [7291] WARNING: there is no transaction in progress
2026-08-06 21:07:36.691 UTC [7291] WARNING: there is no transaction in progress
2026-08-06 21:07:36.707 UTC [7291] WARNING: there is no transaction in progress
2026-08-06 21:07:36.718 UTC [7291] WARNING: there is no transaction in progress
2026-08-06 21:07:36.726 UTC [7291] WARNING: there is no transaction in progress
2026-08-06 21:07:36.735 UTC [7291] WARNING: there is no transaction in progress
2026-08-06 21:07:36.744 UTC [7291] WARNING: there is no transaction in progress
2026-08-06 21:07:37,564 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 21:07:37,565 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:56623/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 21:07:37,814 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 21:07:37,814 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:56623/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 21:07:38,089 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 21:07:38,089 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:56623/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 21:07:38,345 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 21:07:38,345 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:56623/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 21:07:38,600 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:56623/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 21:07:38,866 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:56623/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-69rg.c.gcp-upstream.internal';"
2026-08-06 21:07:39,114 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 21:07:39,115 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:56623/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 21:07:39,384 [I] DB_PATCHES have already been applied
I20260806 21:07:39.390643 28551 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDiceBW/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 21:07:39.390730 28551 mini_ranger.cc:193] Using host: 127.27.225.212
I20260806 21:07:39.395395 28551 mini_ranger.cc:227] Ranger admin URL: http://127.27.225.212:42271
Aug 06, 2026 9:07:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 9:07:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 9:07:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 9:07:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 9:07:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 9:07:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 9:07:40 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 21:07:40.545230 28551 mini_ranger.cc:160] Time spent starting Ranger: real 9.268s user 0.003s sys 0.011s
Aug 06, 2026 9:08:01 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 9:08:01 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 9:08:01 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 9:08:01 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 9:08:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 9:08:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 9:08:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 9:08:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 9:08:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 9:08:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 9:08:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 9:08:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 9:08:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 9:08:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 9:08:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 9:08:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 9:08:01 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 9:08:02 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 21:08:03.420809 28551 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.27.225.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.27.225.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.27.225.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/rangerkms_127.27.225.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.27.225.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/rangerkms_127.27.225.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.27.225.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.27.225.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.27.225.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/HTTP_127.27.225.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.27.225.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/HTTP_127.27.225.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 21:08:03 dist-test-slave-69rg krb5kdc[7074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050483, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 21:08:03.520927 28551 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 21:08:03.533612 28551 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 21:08:04.447502 28551 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 21:08:04.447620 28551 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ranger-kms
I20260806 21:08:04.447734 28551 mini_ranger_kms.cc:227] Using postgres at 127.27.225.212:56623
2026-08-06 21:08:04,747 [I] DB FLAVOR :POSTGRES
2026-08-06 21:08:04,748 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 21:08:04,748 [I] Checking connection
2026-08-06 21:08:04,748 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:56623/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 21:08:05,015 [I] connection success
2026-08-06 21:08:05,015 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 21:08:05,016 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 21:08:05,016 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:56623/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 21:08:05,305 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 21:08:05,306 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 21:08:05,306 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 21:08:05,306 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:56623/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 21:08:05,634 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 21:08:06.198186 28551 mini_ranger_kms.cc:310] Created kms service
I20260806 21:08:06.388679 28551 mini_ranger_kms.cc:326] Created kudu user
I20260806 21:08:06.455574 28551 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 21:08:06 dist-test-slave-69rg krb5kdc[7074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050483, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.27.225.212@KRBTEST.COM
I20260806 21:08:06.944851 28551 mini_ranger_kms.cc:384] Added ranger policy
I20260806 21:08:06.945020 28551 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 21:08:06.945046 28551 mini_ranger_kms.cc:240] Using host: 127.27.225.212
I20260806 21:08:06.949901 28551 mini_ranger_kms.cc:276] Ranger KMS PID: 7477
I20260806 21:08:06.950060 28551 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.27.225.212:39425
Aug 06, 2026 9:08:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 9:08:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 9:08:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 9:08:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 9:08:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 9:08:08 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 21:08:09.331432 28551 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.811s user 0.028s sys 0.258s
I20260806 21:08:09.331598 28551 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 21:08:09.331744 28551 mini_ranger_kms.cc:401] 127.27.225.212:39425/kms/v1/keys
Aug 06 21:08:09 dist-test-slave-69rg krb5kdc[7074](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786050489, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.225.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 21:08:10 dist-test-slave-69rg krb5kdc[7074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050489, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.225.212@KRBTEST.COM for HTTP/127.27.225.212@KRBTEST.COM
Aug 06 21:08:10 dist-test-slave-69rg krb5kdc[7074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050489, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.225.212@KRBTEST.COM for HTTP/127.27.225.212@KRBTEST.COM
Aug 06 21:08:12 dist-test-slave-69rg krb5kdc[7074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:08:13.002954 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:44479
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:34351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:44479
--ranger_config_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.225.212:39425/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:08:13.126993 7529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:08:13.127396 7529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:08:13.127481 7529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:08:13.132056 7529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:08:13.132182 7529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:08:13.132220 7529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:08:13.132277 7529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:08:13.132335 7529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:08:13.139181 7529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:34351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/master-0/wal
--ranger_kms_url=127.27.225.212:39425/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:44479
--ranger_config_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:44479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:08:13.140473 7529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:08:13.141624 7529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:08:13.151718 7537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:08:13.151724 7535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:08:13.151738 7534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:08:13.151889 7529 server_base.cc:1034] running on GCE node
I20260806 21:08:13.152824 7529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:08:13.154296 7529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:08:13.155541 7529 hybrid_clock.cc:648] HybridClock initialized: now 1786050493155495 us; error 75 us; skew 500 ppm
Aug 06 21:08:13 dist-test-slave-69rg krb5kdc[7074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050493, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:08:13.159053 7529 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:08:13.160140 7529 webserver.cc:460] Webserver started at http://127.27.225.254:33107/ using document root <none> and password file <none>
I20260806 21:08:13.160748 7529 fs_manager.cc:359] Metadata directory not provided
I20260806 21:08:13.160836 7529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:08:13.161101 7529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 21:08:13 dist-test-slave-69rg krb5kdc[7074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050493, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for HTTP/127.27.225.212@KRBTEST.COM
I20260806 21:08:13.223847 7529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "789eb9bbc6704497b76ee2895bde2476"
format_stamp: "Formatted at 2026-08-06 21:08:13 on dist-test-slave-69rg"
server_key: "abe152ff1df31148f440422c6702e40d"
server_key_iv: "51523422243096aacd85170500d04a2d"
server_key_version: "kuduclusterkey@0"
I20260806 21:08:13.224529 7529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "789eb9bbc6704497b76ee2895bde2476"
format_stamp: "Formatted at 2026-08-06 21:08:13 on dist-test-slave-69rg"
server_key: "abe152ff1df31148f440422c6702e40d"
server_key_iv: "51523422243096aacd85170500d04a2d"
server_key_version: "kuduclusterkey@0"
I20260806 21:08:13.228871 7529 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260806 21:08:13.247022 7545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:08:13.248737 7529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 21:08:13.248932 7529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "789eb9bbc6704497b76ee2895bde2476"
format_stamp: "Formatted at 2026-08-06 21:08:13 on dist-test-slave-69rg"
server_key: "abe152ff1df31148f440422c6702e40d"
server_key_iv: "51523422243096aacd85170500d04a2d"
server_key_version: "kuduclusterkey@0"
I20260806 21:08:13.249094 7529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:08:13.270424 7529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:08:13.273916 7529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:08:13.274166 7529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:08:13.283021 7529 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:44479
I20260806 21:08:13.283043 7597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:44479 every 8 connection(s)
I20260806 21:08:13.284190 7529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:08:13.287350 7598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:08:13.293686 7598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 789eb9bbc6704497b76ee2895bde2476: Bootstrap starting.
I20260806 21:08:13.293955 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 7529
I20260806 21:08:13.294096 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:08:13.294410 28551 external_mini_cluster.cc:1428] Setting key 81cb78d537d93b62de6a68064d28ce27
I20260806 21:08:13.296685 7598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 789eb9bbc6704497b76ee2895bde2476: Neither blocks nor log segments found. Creating new log.
I20260806 21:08:13.297474 7598 log.cc:824] T 00000000000000000000000000000000 P 789eb9bbc6704497b76ee2895bde2476: Log is configured to *not* fsync() on all Append() calls
I20260806 21:08:13.299755 7598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 789eb9bbc6704497b76ee2895bde2476: No bootstrap required, opened a new log
Aug 06 21:08:13 dist-test-slave-69rg krb5kdc[7074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:08:13.304879 7598 raft_consensus.cc:348] T 00000000000000000000000000000000 P 789eb9bbc6704497b76ee2895bde2476 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "789eb9bbc6704497b76ee2895bde2476" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44479 } }
I20260806 21:08:13.305166 7598 raft_consensus.cc:374] T 00000000000000000000000000000000 P 789eb9bbc6704497b76ee2895bde2476 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:08:13.305240 7598 raft_consensus.cc:729] T 00000000000000000000000000000000 P 789eb9bbc6704497b76ee2895bde2476 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 789eb9bbc6704497b76ee2895bde2476, State: Initialized, Role: FOLLOWER
I20260806 21:08:13.305711 7598 consensus_queue.cc:260] T 00000000000000000000000000000000 P 789eb9bbc6704497b76ee2895bde2476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "789eb9bbc6704497b76ee2895bde2476" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44479 } }
I20260806 21:08:13.305879 7598 raft_consensus.cc:388] T 00000000000000000000000000000000 P 789eb9bbc6704497b76ee2895bde2476 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:08:13.305971 7598 raft_consensus.cc:482] T 00000000000000000000000000000000 P 789eb9bbc6704497b76ee2895bde2476 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:08:13.306114 7598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 789eb9bbc6704497b76ee2895bde2476 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:08:13.307142 7598 raft_consensus.cc:504] T 00000000000000000000000000000000 P 789eb9bbc6704497b76ee2895bde2476 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "789eb9bbc6704497b76ee2895bde2476" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44479 } }
I20260806 21:08:13.307559 7598 leader_election.cc:302] T 00000000000000000000000000000000 P 789eb9bbc6704497b76ee2895bde2476 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 789eb9bbc6704497b76ee2895bde2476; no voters:
I20260806 21:08:13.307920 7598 leader_election.cc:288] T 00000000000000000000000000000000 P 789eb9bbc6704497b76ee2895bde2476 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:08:13.308038 7603 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 789eb9bbc6704497b76ee2895bde2476 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:08:13.308267 7603 raft_consensus.cc:686] T 00000000000000000000000000000000 P 789eb9bbc6704497b76ee2895bde2476 [term 1 LEADER]: Becoming Leader. State: Replica: 789eb9bbc6704497b76ee2895bde2476, State: Running, Role: LEADER
I20260806 21:08:13.308645 7603 consensus_queue.cc:237] T 00000000000000000000000000000000 P 789eb9bbc6704497b76ee2895bde2476 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "789eb9bbc6704497b76ee2895bde2476" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44479 } }
I20260806 21:08:13.309311 7598 sys_catalog.cc:564] T 00000000000000000000000000000000 P 789eb9bbc6704497b76ee2895bde2476 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:08:13.310310 7601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:08:13.295958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37686 (local address 127.27.225.254:44479)
0806 21:08:13.296460 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0806 21:08:13.296476 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:08:13.296512 (+ 36us) server_negotiation.cc:407] Connection header received
0806 21:08:13.297263 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:08:13.297284 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:08:13.297689 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:08:13.298107 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:08:13.298991 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:08:13.300071 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:08:13.301261 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:08:13.301747 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:08:13.301981 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:08:13.305037 (+ 3056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:08:13.305043 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:08:13.305056 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:08:13.305084 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:08:13.306988 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:08:13.307504 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:08:13.307508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:08:13.307513 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:08:13.307618 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:08:13.307968 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:08:13.307971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:08:13.307972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:08:13.308537 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:08:13.308724 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 21:08:13.308997 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 21:08:13.309318 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":164,"threads_started":1}
I20260806 21:08:13.310474 7604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 789eb9bbc6704497b76ee2895bde2476 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "789eb9bbc6704497b76ee2895bde2476" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "789eb9bbc6704497b76ee2895bde2476" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44479 } } }
I20260806 21:08:13.310599 7604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 789eb9bbc6704497b76ee2895bde2476 [sys.catalog]: This master's current role is: LEADER
I20260806 21:08:13.311798 7605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 789eb9bbc6704497b76ee2895bde2476 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 789eb9bbc6704497b76ee2895bde2476. Latest consensus state: current_term: 1 leader_uuid: "789eb9bbc6704497b76ee2895bde2476" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "789eb9bbc6704497b76ee2895bde2476" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44479 } } }
I20260806 21:08:13.311838 7598 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 21:08:13.311976 7605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 789eb9bbc6704497b76ee2895bde2476 [sys.catalog]: This master's current role is: LEADER
I20260806 21:08:13.313578 7612 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:08:13.321305 7612 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:08:13.329154 7612 catalog_manager.cc:1347] Generated new cluster ID: 0ed36faf333e44eabb89131d76683a50
I20260806 21:08:13.329344 7612 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:08:13.356168 7612 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:08:13.357640 7612 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:08:13.372725 7612 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 789eb9bbc6704497b76ee2895bde2476: Generated new TSK 0
I20260806 21:08:13.373682 7612 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.225.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
I20260806 21:08:14.138149 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.225.212:39425/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:44479
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:34351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:08:14.301535 7645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:08:14.302021 7645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:08:14.302141 7645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:08:14.308676 7645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:08:14.308768 7645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:08:14.308876 7645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:08:14.315465 7645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:34351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.27.225.212:39425/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:44479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:08:14.316633 7645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:08:14.317775 7645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:08:14.328459 7655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:08:14.328459 7653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:08:14.328769 7652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:08:14.329514 7645 server_base.cc:1034] running on GCE node
I20260806 21:08:14.330087 7645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:08:14.330813 7645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:08:14.332013 7645 hybrid_clock.cc:648] HybridClock initialized: now 1786050494331980 us; error 54 us; skew 500 ppm
Aug 06 21:08:14 dist-test-slave-69rg krb5kdc[7074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050494, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:08:14.335105 7645 init.cc:377] Logged in from keytab as kudu/127.27.225.193@KRBTEST.COM (short username kudu)
I20260806 21:08:14.336156 7645 webserver.cc:460] Webserver started at http://127.27.225.193:42227/ using document root <none> and password file <none>
I20260806 21:08:14.336733 7645 fs_manager.cc:359] Metadata directory not provided
I20260806 21:08:14.336824 7645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:08:14.337035 7645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 21:08:14 dist-test-slave-69rg krb5kdc[7074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050494, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for HTTP/127.27.225.212@KRBTEST.COM
I20260806 21:08:14.356995 7645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "917b25b34b5f4aa2b8e988f3fa509339"
format_stamp: "Formatted at 2026-08-06 21:08:14 on dist-test-slave-69rg"
server_key: "df567a921ad71fc9cc10b765d8932054"
server_key_iv: "f63b8e23073b7ad613bfd31628a0918e"
server_key_version: "kuduclusterkey@0"
I20260806 21:08:14.357779 7645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "917b25b34b5f4aa2b8e988f3fa509339"
format_stamp: "Formatted at 2026-08-06 21:08:14 on dist-test-slave-69rg"
server_key: "df567a921ad71fc9cc10b765d8932054"
server_key_iv: "f63b8e23073b7ad613bfd31628a0918e"
server_key_version: "kuduclusterkey@0"
I20260806 21:08:14.363859 7645 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 21:08:14.381210 7662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:08:14.382978 7645 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:08:14.383239 7645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "917b25b34b5f4aa2b8e988f3fa509339"
format_stamp: "Formatted at 2026-08-06 21:08:14 on dist-test-slave-69rg"
server_key: "df567a921ad71fc9cc10b765d8932054"
server_key_iv: "f63b8e23073b7ad613bfd31628a0918e"
server_key_version: "kuduclusterkey@0"
I20260806 21:08:14.383487 7645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:08:14.403010 7645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:08:14.407377 7645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:08:14.407689 7645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:08:14.408545 7645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:08:14.409914 7645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:08:14.410033 7645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:08:14.410190 7645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:08:14.410240 7645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:08:14.422430 7645 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:38305
I20260806 21:08:14.423822 7645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:08:14.426601 7781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:38305 every 8 connection(s)
I20260806 21:08:14.429733 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 7645
I20260806 21:08:14.429893 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:08:14.430272 28551 external_mini_cluster.cc:1428] Setting key f57c50b830fd35e3e63a9d4ff2b90a7e
Aug 06 21:08:14 dist-test-slave-69rg krb5kdc[7074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050494, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:08:14.456555 7785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:08:14.429349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:44479 (local address 127.27.225.193:45461)
0806 21:08:14.439035 (+ 9686us) negotiation.cc:107] Waiting for socket to connect
0806 21:08:14.439134 (+ 99us) client_negotiation.cc:174] Beginning negotiation
0806 21:08:14.440543 (+ 1409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:08:14.441949 (+ 1406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:08:14.441961 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:08:14.442394 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:08:14.443112 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:08:14.443132 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:08:14.444439 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:08:14.444443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:08:14.445999 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:08:14.446018 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:08:14.446982 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:08:14.446989 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:08:14.447274 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:08:14.448126 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:08:14.448151 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:08:14.450188 (+ 2037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:08:14.452610 (+ 2422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:08:14.452618 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:08:14.452638 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:08:14.453127 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:08:14.453518 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:08:14.453524 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:08:14.453527 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:08:14.453738 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:08:14.454433 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:08:14.454440 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:08:14.454809 (+ 369us) client_negotiation.cc:769] Sending connection context
0806 21:08:14.455009 (+ 200us) client_negotiation.cc:240] Negotiation successful
0806 21:08:14.455381 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9341,"thread_start_us":143,"threads_started":1}
I20260806 21:08:14.457436 7784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:08:14.428789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:45461 (local address 127.27.225.254:44479)
0806 21:08:14.441071 (+ 12282us) server_negotiation.cc:206] Beginning negotiation
0806 21:08:14.441078 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:08:14.441097 (+ 19us) server_negotiation.cc:407] Connection header received
0806 21:08:14.441218 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:08:14.441230 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:08:14.441309 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:08:14.441425 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:08:14.443371 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:08:14.444269 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:08:14.446183 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:08:14.446611 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:08:14.446739 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:08:14.450461 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:08:14.450466 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:08:14.450469 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:08:14.450501 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:08:14.452433 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:08:14.453271 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:08:14.453274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:08:14.453277 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:08:14.453378 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:08:14.453885 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:08:14.453889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:08:14.453890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:08:14.454256 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:08:14.454369 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 21:08:14.457093 (+ 2724us) server_negotiation.cc:299] Negotiation successful
0806 21:08:14.457260 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12166,"thread_start_us":62,"threads_started":1}
WARNING: no policy specified for kudu/127.27.225.194@KRBTEST.COM; defaulting to no policy
I20260806 21:08:14.459421 7782 heartbeater.cc:347] Connected to a master server at 127.27.225.254:44479
I20260806 21:08:14.459679 7782 heartbeater.cc:464] Registering TS with master...
I20260806 21:08:14.460301 7782 heartbeater.cc:511] Master 127.27.225.254:44479 requested a full tablet report, sending...
I20260806 21:08:14.462136 7562 ts_manager.cc:194] Registered new tserver with Master: 917b25b34b5f4aa2b8e988f3fa509339 (127.27.225.193:38305)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.194@KRBTEST.COM" created.
I20260806 21:08:14.463946 7562 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.193@KRBTEST.COM'} at 127.27.225.193:45461
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
I20260806 21:08:14.489290 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.225.212:39425/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:44479
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:34351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 21:08:14 dist-test-slave-69rg krb5kdc[7074](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786050494, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 21:08:14.655747 7788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:08:14.656284 7788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:08:14.656385 7788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:08:14.663523 7788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:08:14.663659 7788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:08:14.663756 7788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:08:14.671708 7788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:34351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.27.225.212:39425/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:44479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:08:14.673000 7788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:08:14.674309 7788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:08:14.686621 7797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:08:14.686640 7794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:08:14.686658 7795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:08:14.687883 7788 server_base.cc:1034] running on GCE node
I20260806 21:08:14.688365 7788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:08:14.689167 7788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:08:14.690371 7788 hybrid_clock.cc:648] HybridClock initialized: now 1786050494690340 us; error 45 us; skew 500 ppm
Aug 06 21:08:14 dist-test-slave-69rg krb5kdc[7074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050494, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:08:14.694550 7788 init.cc:377] Logged in from keytab as kudu/127.27.225.194@KRBTEST.COM (short username kudu)
I20260806 21:08:14.695835 7788 webserver.cc:460] Webserver started at http://127.27.225.194:33227/ using document root <none> and password file <none>
I20260806 21:08:14.696609 7788 fs_manager.cc:359] Metadata directory not provided
I20260806 21:08:14.696707 7788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:08:14.696959 7788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 21:08:14 dist-test-slave-69rg krb5kdc[7074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050494, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for HTTP/127.27.225.212@KRBTEST.COM
I20260806 21:08:14.716859 7788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "643a95ad541e4e978737cfd2feaa92d6"
format_stamp: "Formatted at 2026-08-06 21:08:14 on dist-test-slave-69rg"
server_key: "1fd49b32aceccdbece3ef0f6ddf05871"
server_key_iv: "079658180e2d041409214278b8d2becd"
server_key_version: "kuduclusterkey@0"
I20260806 21:08:14.717531 7788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "643a95ad541e4e978737cfd2feaa92d6"
format_stamp: "Formatted at 2026-08-06 21:08:14 on dist-test-slave-69rg"
server_key: "1fd49b32aceccdbece3ef0f6ddf05871"
server_key_iv: "079658180e2d041409214278b8d2becd"
server_key_version: "kuduclusterkey@0"
I20260806 21:08:14.721927 7788 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:08:14.741588 7804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:08:14.743404 7788 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 21:08:14.743549 7788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "643a95ad541e4e978737cfd2feaa92d6"
format_stamp: "Formatted at 2026-08-06 21:08:14 on dist-test-slave-69rg"
server_key: "1fd49b32aceccdbece3ef0f6ddf05871"
server_key_iv: "079658180e2d041409214278b8d2becd"
server_key_version: "kuduclusterkey@0"
I20260806 21:08:14.743698 7788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:08:14.769008 7788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:08:14.773123 7788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:08:14.773438 7788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:08:14.774283 7788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:08:14.775414 7788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:08:14.775521 7788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:08:14.775615 7788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:08:14.775682 7788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:08:14.790689 7788 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:37391
I20260806 21:08:14.792014 7788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:08:14.793431 7920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:37391 every 8 connection(s)
I20260806 21:08:14.800668 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 7788
I20260806 21:08:14.800817 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:08:14.801134 28551 external_mini_cluster.cc:1428] Setting key 35feb11886c6e794e414dadcf7da725b
Aug 06 21:08:14 dist-test-slave-69rg krb5kdc[7074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050494, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.225.195@KRBTEST.COM; defaulting to no policy
I20260806 21:08:14.821107 7784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:08:14.797382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:44803 (local address 127.27.225.254:44479)
0806 21:08:14.797522 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 21:08:14.797527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:08:14.801792 (+ 4265us) server_negotiation.cc:407] Connection header received
0806 21:08:14.807383 (+ 5591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:08:14.807386 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:08:14.807442 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:08:14.807578 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:08:14.809035 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:08:14.809808 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:08:14.811797 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:08:14.812179 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:08:14.812271 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:08:14.815701 (+ 3430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:08:14.815706 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:08:14.815708 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:08:14.815754 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:08:14.817812 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:08:14.818726 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:08:14.818731 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:08:14.818733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:08:14.818845 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:08:14.819326 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:08:14.819330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:08:14.819331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:08:14.819591 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:08:14.819697 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 21:08:14.820572 (+ 875us) server_negotiation.cc:299] Negotiation successful
0806 21:08:14.820919 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 21:08:14.823052 7923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:08:14.796190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:44479 (local address 127.27.225.194:44803)
0806 21:08:14.801573 (+ 5383us) negotiation.cc:107] Waiting for socket to connect
0806 21:08:14.801647 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 21:08:14.807051 (+ 5404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:08:14.807817 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:08:14.807826 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:08:14.808152 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:08:14.808723 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:08:14.808776 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:08:14.810061 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:08:14.810064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:08:14.811594 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:08:14.811611 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:08:14.812530 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:08:14.812534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:08:14.812788 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:08:14.813592 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:08:14.813617 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:08:14.815431 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:08:14.818043 (+ 2612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:08:14.818052 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:08:14.818099 (+ 47us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:08:14.818556 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:08:14.818969 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:08:14.818971 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:08:14.818974 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:08:14.819206 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:08:14.819796 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:08:14.819804 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:08:14.820236 (+ 432us) client_negotiation.cc:769] Sending connection context
0806 21:08:14.821317 (+ 1081us) client_negotiation.cc:240] Negotiation successful
0806 21:08:14.821780 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5100,"thread_start_us":164,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.195@KRBTEST.COM" created.
I20260806 21:08:14.825152 7921 heartbeater.cc:347] Connected to a master server at 127.27.225.254:44479
I20260806 21:08:14.825436 7921 heartbeater.cc:464] Registering TS with master...
I20260806 21:08:14.826112 7921 heartbeater.cc:511] Master 127.27.225.254:44479 requested a full tablet report, sending...
I20260806 21:08:14.827473 7562 ts_manager.cc:194] Registered new tserver with Master: 643a95ad541e4e978737cfd2feaa92d6 (127.27.225.194:37391)
I20260806 21:08:14.828239 7562 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.194@KRBTEST.COM'} at 127.27.225.194:44803
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
I20260806 21:08:14.859597 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.225.212:39425/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:44479
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:34351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:08:15.030181 7926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:08:15.030643 7926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:08:15.030766 7926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:08:15.037216 7926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:08:15.037391 7926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:08:15.037533 7926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:08:15.046360 7926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:34351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.27.225.212:39425/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:44479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:08:15.048122 7926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:08:15.049682 7926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:08:15.063545 7934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:08:15.063839 7932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:08:15.063541 7931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:08:15.065173 7926 server_base.cc:1034] running on GCE node
I20260806 21:08:15.065689 7926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:08:15.066545 7926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:08:15.085067 7926 hybrid_clock.cc:648] HybridClock initialized: now 1786050495084937 us; error 126 us; skew 500 ppm
Aug 06 21:08:15 dist-test-slave-69rg krb5kdc[7074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050495, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:08:15.089291 7926 init.cc:377] Logged in from keytab as kudu/127.27.225.195@KRBTEST.COM (short username kudu)
I20260806 21:08:15.090595 7926 webserver.cc:460] Webserver started at http://127.27.225.195:38461/ using document root <none> and password file <none>
I20260806 21:08:15.091359 7926 fs_manager.cc:359] Metadata directory not provided
I20260806 21:08:15.091513 7926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:08:15.091790 7926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 21:08:15 dist-test-slave-69rg krb5kdc[7074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050495, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for HTTP/127.27.225.212@KRBTEST.COM
Aug 06 21:08:15 dist-test-slave-69rg krb5kdc[7074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050494, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for HTTP/127.27.225.212@KRBTEST.COM
I20260806 21:08:15.110767 7926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "b4cb11b97c5f46f39cac5cb710582058"
format_stamp: "Formatted at 2026-08-06 21:08:15 on dist-test-slave-69rg"
server_key: "65b8a6e09bc25c30be97d836be434705"
server_key_iv: "14c2e9a37c6eb36a766a856c0a45cd5b"
server_key_version: "kuduclusterkey@0"
I20260806 21:08:15.112674 7926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "b4cb11b97c5f46f39cac5cb710582058"
format_stamp: "Formatted at 2026-08-06 21:08:15 on dist-test-slave-69rg"
server_key: "65b8a6e09bc25c30be97d836be434705"
server_key_iv: "14c2e9a37c6eb36a766a856c0a45cd5b"
server_key_version: "kuduclusterkey@0"
I20260806 21:08:15.118642 7926 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 21:08:15.152344 7941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:08:15.154691 7926 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.003s sys 0.000s
I20260806 21:08:15.154857 7926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "b4cb11b97c5f46f39cac5cb710582058"
format_stamp: "Formatted at 2026-08-06 21:08:15 on dist-test-slave-69rg"
server_key: "65b8a6e09bc25c30be97d836be434705"
server_key_iv: "14c2e9a37c6eb36a766a856c0a45cd5b"
server_key_version: "kuduclusterkey@0"
I20260806 21:08:15.155031 7926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:08:15.193616 7926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:08:15.197562 7926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:08:15.197856 7926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:08:15.198916 7926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:08:15.200006 7926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:08:15.200116 7926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:08:15.200233 7926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:08:15.200309 7926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:08:15.222997 7926 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:34145
I20260806 21:08:15.224295 7926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:08:15.226490 8057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:34145 every 8 connection(s)
I20260806 21:08:15.231390 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 7926
I20260806 21:08:15.231520 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:08:15.231830 28551 external_mini_cluster.cc:1428] Setting key 4f928ccab1e8761a94bdf21c94696d2f
Aug 06 21:08:15 dist-test-slave-69rg krb5kdc[7074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050495, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:08:15.249754 7784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:08:15.228582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:35281 (local address 127.27.225.254:44479)
0806 21:08:15.228724 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 21:08:15.228729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:08:15.232912 (+ 4183us) server_negotiation.cc:407] Connection header received
0806 21:08:15.235608 (+ 2696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:08:15.235611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:08:15.235667 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:08:15.235717 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:08:15.237585 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.238470 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.240243 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.240616 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.240705 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:08:15.244111 (+ 3406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:08:15.244116 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:08:15.244119 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:08:15.244148 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:08:15.246046 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:08:15.247280 (+ 1234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:08:15.247285 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:08:15.247287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:08:15.247394 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:08:15.248006 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:08:15.248010 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:08:15.248012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:08:15.248354 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:08:15.248441 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 21:08:15.249376 (+ 935us) server_negotiation.cc:299] Negotiation successful
0806 21:08:15.249562 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 21:08:15.251997 8060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:08:15.229466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:44479 (local address 127.27.225.195:35281)
0806 21:08:15.232658 (+ 3192us) negotiation.cc:107] Waiting for socket to connect
0806 21:08:15.232725 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 21:08:15.235338 (+ 2613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:08:15.236161 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:08:15.236172 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:08:15.236572 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:08:15.237300 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:08:15.237322 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.238653 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.238657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:08:15.240080 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:08:15.240093 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.240939 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.240944 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:08:15.241168 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:08:15.241978 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:08:15.242000 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:08:15.243880 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:08:15.246491 (+ 2611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:08:15.246498 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:08:15.246517 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:08:15.247117 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:08:15.247540 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:08:15.247545 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:08:15.247550 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:08:15.247875 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:08:15.248540 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:08:15.248549 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:08:15.248936 (+ 387us) client_negotiation.cc:769] Sending connection context
0806 21:08:15.250359 (+ 1423us) client_negotiation.cc:240] Negotiation successful
0806 21:08:15.250771 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1513,"thread_start_us":159,"threads_started":1}
I20260806 21:08:15.253736 8058 heartbeater.cc:347] Connected to a master server at 127.27.225.254:44479
I20260806 21:08:15.254016 8058 heartbeater.cc:464] Registering TS with master...
I20260806 21:08:15.254662 8058 heartbeater.cc:511] Master 127.27.225.254:44479 requested a full tablet report, sending...
I20260806 21:08:15.255870 7562 ts_manager.cc:194] Registered new tserver with Master: b4cb11b97c5f46f39cac5cb710582058 (127.27.225.195:34145)
I20260806 21:08:15.256587 7562 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.195@KRBTEST.COM'} at 127.27.225.195:35281
I20260806 21:08:15.258785 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:08:15.270051 7784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:08:15.261285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37688 (local address 127.27.225.254:44479)
0806 21:08:15.261426 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 21:08:15.261432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:08:15.261665 (+ 233us) server_negotiation.cc:407] Connection header received
0806 21:08:15.261803 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:08:15.261806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:08:15.261875 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:08:15.261957 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:08:15.262792 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.263516 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.264442 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.264735 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.264841 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:08:15.266453 (+ 1612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:08:15.266456 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:08:15.266459 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:08:15.266487 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:08:15.268251 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:08:15.268747 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:08:15.268750 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:08:15.268751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:08:15.268831 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:08:15.269123 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:08:15.269126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:08:15.269127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:08:15.269309 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:08:15.269417 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 21:08:15.269702 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 21:08:15.269806 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 21:08:15.273288 7562 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:08:15.277429 7562 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:08:15.305173 8071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:08:15.294158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:37391 (local address 127.0.0.1:60648)
0806 21:08:15.294644 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0806 21:08:15.294705 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 21:08:15.294842 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:08:15.298505 (+ 3663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:08:15.298508 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:08:15.298526 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:08:15.298618 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:08:15.298634 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.300158 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.300161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:08:15.301569 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:08:15.301582 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.303049 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.303052 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:08:15.304421 (+ 1369us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:08:15.304489 (+ 68us) client_negotiation.cc:769] Sending connection context
0806 21:08:15.304609 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 21:08:15.304900 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":1203,"thread_start_us":135,"threads_started":1}
I20260806 21:08:15.306245 8072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:08:15.294272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60648 (local address 127.27.225.194:37391)
0806 21:08:15.297797 (+ 3525us) server_negotiation.cc:206] Beginning negotiation
0806 21:08:15.297841 (+ 44us) server_negotiation.cc:399] Waiting for connection header
0806 21:08:15.297869 (+ 28us) server_negotiation.cc:407] Connection header received
0806 21:08:15.297950 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:08:15.297955 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:08:15.298183 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:08:15.298342 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:08:15.298767 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.299992 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.301775 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.302701 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.302832 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:08:15.302979 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 21:08:15.305937 (+ 2958us) server_negotiation.cc:299] Negotiation successful
0806 21:08:15.306083 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3411,"thread_start_us":125,"threads_started":1}
I20260806 21:08:15.306457 8073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:08:15.295550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:38305 (local address 127.0.0.1:59902)
0806 21:08:15.298170 (+ 2620us) negotiation.cc:107] Waiting for socket to connect
0806 21:08:15.298186 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:08:15.298376 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:08:15.299134 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:08:15.299140 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:08:15.299163 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:08:15.299267 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:08:15.299272 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.302290 (+ 3018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.302294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:08:15.303522 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:08:15.303535 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.305988 (+ 2453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.305991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:08:15.306098 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:08:15.306128 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 21:08:15.306220 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 21:08:15.306305 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2550,"mutex_wait_us":4,"thread_start_us":101,"threads_started":1}
I20260806 21:08:15.307039 8076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:08:15.295761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59902 (local address 127.27.225.193:38305)
0806 21:08:15.297536 (+ 1775us) server_negotiation.cc:206] Beginning negotiation
0806 21:08:15.297545 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 21:08:15.298279 (+ 734us) server_negotiation.cc:407] Connection header received
0806 21:08:15.298537 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:08:15.298544 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:08:15.298735 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:08:15.298837 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:08:15.299397 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.302129 (+ 2732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.303685 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.305584 (+ 1899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.305690 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:08:15.305874 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 21:08:15.306796 (+ 922us) server_negotiation.cc:299] Negotiation successful
0806 21:08:15.306905 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1651,"thread_start_us":113,"threads_started":1}
I20260806 21:08:15.309687 8075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:08:15.295072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:34145 (local address 127.0.0.1:36350)
0806 21:08:15.296158 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0806 21:08:15.296176 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:08:15.296296 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:08:15.297373 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:08:15.297381 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:08:15.297409 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:08:15.297589 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:08:15.297599 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.306678 (+ 9079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.306682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:08:15.307943 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:08:15.307956 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.309277 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.309281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:08:15.309390 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:08:15.309418 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 21:08:15.309475 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 21:08:15.309548 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1001,"thread_start_us":101,"threads_started":1}
I20260806 21:08:15.309803 7852 tablet_service.cc:1467] Processing CreateTablet for tablet e183930f1eb842bf937b5f5ff49b3391 (DEFAULT_TABLE table=test-table [id=1e3c452fd3f246fe8eee877bfde39b76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:08:15.309803 7715 tablet_service.cc:1467] Processing CreateTablet for tablet e183930f1eb842bf937b5f5ff49b3391 (DEFAULT_TABLE table=test-table [id=1e3c452fd3f246fe8eee877bfde39b76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:08:15.310881 8074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:08:15.295251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36350 (local address 127.27.225.195:34145)
0806 21:08:15.296665 (+ 1414us) server_negotiation.cc:206] Beginning negotiation
0806 21:08:15.296680 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 21:08:15.296710 (+ 30us) server_negotiation.cc:407] Connection header received
0806 21:08:15.296803 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:08:15.296809 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:08:15.297082 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:08:15.297302 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:08:15.297764 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.304225 (+ 6461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.308180 (+ 3955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.309056 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.310296 (+ 1240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:08:15.310564 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0806 21:08:15.310621 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 21:08:15.310725 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1207,"thread_start_us":150,"threads_started":1}
I20260806 21:08:15.311020 7715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e183930f1eb842bf937b5f5ff49b3391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:08:15.311020 7852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e183930f1eb842bf937b5f5ff49b3391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:08:15.314090 7991 tablet_service.cc:1467] Processing CreateTablet for tablet e183930f1eb842bf937b5f5ff49b3391 (DEFAULT_TABLE table=test-table [id=1e3c452fd3f246fe8eee877bfde39b76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:08:15.315274 7991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e183930f1eb842bf937b5f5ff49b3391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:08:15.321566 8078 tablet_bootstrap.cc:492] T e183930f1eb842bf937b5f5ff49b3391 P 917b25b34b5f4aa2b8e988f3fa509339: Bootstrap starting.
I20260806 21:08:15.324169 8078 tablet_bootstrap.cc:654] T e183930f1eb842bf937b5f5ff49b3391 P 917b25b34b5f4aa2b8e988f3fa509339: Neither blocks nor log segments found. Creating new log.
I20260806 21:08:15.324587 8077 tablet_bootstrap.cc:492] T e183930f1eb842bf937b5f5ff49b3391 P 643a95ad541e4e978737cfd2feaa92d6: Bootstrap starting.
I20260806 21:08:15.326242 8078 log.cc:824] T e183930f1eb842bf937b5f5ff49b3391 P 917b25b34b5f4aa2b8e988f3fa509339: Log is configured to *not* fsync() on all Append() calls
I20260806 21:08:15.327162 8077 tablet_bootstrap.cc:654] T e183930f1eb842bf937b5f5ff49b3391 P 643a95ad541e4e978737cfd2feaa92d6: Neither blocks nor log segments found. Creating new log.
I20260806 21:08:15.328722 8078 tablet_bootstrap.cc:492] T e183930f1eb842bf937b5f5ff49b3391 P 917b25b34b5f4aa2b8e988f3fa509339: No bootstrap required, opened a new log
I20260806 21:08:15.328961 8078 ts_tablet_manager.cc:1397] T e183930f1eb842bf937b5f5ff49b3391 P 917b25b34b5f4aa2b8e988f3fa509339: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260806 21:08:15.329320 8079 tablet_bootstrap.cc:492] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058: Bootstrap starting.
I20260806 21:08:15.329533 8077 log.cc:824] T e183930f1eb842bf937b5f5ff49b3391 P 643a95ad541e4e978737cfd2feaa92d6: Log is configured to *not* fsync() on all Append() calls
I20260806 21:08:15.333104 8079 tablet_bootstrap.cc:654] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058: Neither blocks nor log segments found. Creating new log.
I20260806 21:08:15.334242 8079 log.cc:824] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058: Log is configured to *not* fsync() on all Append() calls
I20260806 21:08:15.336220 8078 raft_consensus.cc:348] T e183930f1eb842bf937b5f5ff49b3391 P 917b25b34b5f4aa2b8e988f3fa509339 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643a95ad541e4e978737cfd2feaa92d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37391 } } peers { permanent_uuid: "b4cb11b97c5f46f39cac5cb710582058" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34145 } } peers { permanent_uuid: "917b25b34b5f4aa2b8e988f3fa509339" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 38305 } }
I20260806 21:08:15.336548 8078 raft_consensus.cc:374] T e183930f1eb842bf937b5f5ff49b3391 P 917b25b34b5f4aa2b8e988f3fa509339 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:08:15.336608 8078 raft_consensus.cc:729] T e183930f1eb842bf937b5f5ff49b3391 P 917b25b34b5f4aa2b8e988f3fa509339 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 917b25b34b5f4aa2b8e988f3fa509339, State: Initialized, Role: FOLLOWER
I20260806 21:08:15.337188 8078 consensus_queue.cc:260] T e183930f1eb842bf937b5f5ff49b3391 P 917b25b34b5f4aa2b8e988f3fa509339 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643a95ad541e4e978737cfd2feaa92d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37391 } } peers { permanent_uuid: "b4cb11b97c5f46f39cac5cb710582058" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34145 } } peers { permanent_uuid: "917b25b34b5f4aa2b8e988f3fa509339" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 38305 } }
I20260806 21:08:15.338478 8078 ts_tablet_manager.cc:1428] T e183930f1eb842bf937b5f5ff49b3391 P 917b25b34b5f4aa2b8e988f3fa509339: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260806 21:08:15.338862 7782 heartbeater.cc:503] Master 127.27.225.254:44479 was elected leader, sending a full tablet report...
I20260806 21:08:15.339157 8077 tablet_bootstrap.cc:492] T e183930f1eb842bf937b5f5ff49b3391 P 643a95ad541e4e978737cfd2feaa92d6: No bootstrap required, opened a new log
I20260806 21:08:15.339373 8077 ts_tablet_manager.cc:1397] T e183930f1eb842bf937b5f5ff49b3391 P 643a95ad541e4e978737cfd2feaa92d6: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.000s
I20260806 21:08:15.343297 8077 raft_consensus.cc:348] T e183930f1eb842bf937b5f5ff49b3391 P 643a95ad541e4e978737cfd2feaa92d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643a95ad541e4e978737cfd2feaa92d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37391 } } peers { permanent_uuid: "b4cb11b97c5f46f39cac5cb710582058" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34145 } } peers { permanent_uuid: "917b25b34b5f4aa2b8e988f3fa509339" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 38305 } }
I20260806 21:08:15.343623 8077 raft_consensus.cc:374] T e183930f1eb842bf937b5f5ff49b3391 P 643a95ad541e4e978737cfd2feaa92d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:08:15.343672 8077 raft_consensus.cc:729] T e183930f1eb842bf937b5f5ff49b3391 P 643a95ad541e4e978737cfd2feaa92d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643a95ad541e4e978737cfd2feaa92d6, State: Initialized, Role: FOLLOWER
I20260806 21:08:15.344141 8077 consensus_queue.cc:260] T e183930f1eb842bf937b5f5ff49b3391 P 643a95ad541e4e978737cfd2feaa92d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643a95ad541e4e978737cfd2feaa92d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37391 } } peers { permanent_uuid: "b4cb11b97c5f46f39cac5cb710582058" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34145 } } peers { permanent_uuid: "917b25b34b5f4aa2b8e988f3fa509339" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 38305 } }
I20260806 21:08:15.344877 7921 heartbeater.cc:503] Master 127.27.225.254:44479 was elected leader, sending a full tablet report...
I20260806 21:08:15.345271 8077 ts_tablet_manager.cc:1428] T e183930f1eb842bf937b5f5ff49b3391 P 643a95ad541e4e978737cfd2feaa92d6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 21:08:15.348629 8079 tablet_bootstrap.cc:492] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058: No bootstrap required, opened a new log
I20260806 21:08:15.348850 8079 ts_tablet_manager.cc:1397] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.000s
I20260806 21:08:15.354094 8079 raft_consensus.cc:348] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643a95ad541e4e978737cfd2feaa92d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37391 } } peers { permanent_uuid: "b4cb11b97c5f46f39cac5cb710582058" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34145 } } peers { permanent_uuid: "917b25b34b5f4aa2b8e988f3fa509339" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 38305 } }
I20260806 21:08:15.354425 8079 raft_consensus.cc:374] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:08:15.354487 8079 raft_consensus.cc:729] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4cb11b97c5f46f39cac5cb710582058, State: Initialized, Role: FOLLOWER
I20260806 21:08:15.355026 8079 consensus_queue.cc:260] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643a95ad541e4e978737cfd2feaa92d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37391 } } peers { permanent_uuid: "b4cb11b97c5f46f39cac5cb710582058" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34145 } } peers { permanent_uuid: "917b25b34b5f4aa2b8e988f3fa509339" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 38305 } }
I20260806 21:08:15.355790 8058 heartbeater.cc:503] Master 127.27.225.254:44479 was elected leader, sending a full tablet report...
I20260806 21:08:15.357366 8079 ts_tablet_manager.cc:1428] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058: Time spent starting tablet: real 0.008s user 0.000s sys 0.006s
W20260806 21:08:15.427995 7783 tablet.cc:2369] T e183930f1eb842bf937b5f5ff49b3391 P 917b25b34b5f4aa2b8e988f3fa509339: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 21:08:15 dist-test-slave-69rg krb5kdc[7074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050494, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for HTTP/127.27.225.212@KRBTEST.COM
I20260806 21:08:15.465667 8086 raft_consensus.cc:482] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:08:15.465946 8086 raft_consensus.cc:504] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643a95ad541e4e978737cfd2feaa92d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37391 } } peers { permanent_uuid: "b4cb11b97c5f46f39cac5cb710582058" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34145 } } peers { permanent_uuid: "917b25b34b5f4aa2b8e988f3fa509339" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 38305 } }
I20260806 21:08:15.467339 8086 leader_election.cc:288] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643a95ad541e4e978737cfd2feaa92d6 (127.27.225.194:37391), 917b25b34b5f4aa2b8e988f3fa509339 (127.27.225.193:38305)
I20260806 21:08:15.472721 8060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:08:15.468140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:37391 (local address 127.27.225.195:39379)
0806 21:08:15.468287 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 21:08:15.468307 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 21:08:15.468416 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:08:15.469285 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:08:15.469288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:08:15.469310 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:08:15.469426 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:08:15.469433 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.470509 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.470513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:08:15.471299 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:08:15.471308 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.472283 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.472287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:08:15.472394 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:08:15.472419 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 21:08:15.472496 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 21:08:15.472576 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 21:08:15.473845 8072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:08:15.467873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:39379 (local address 127.27.225.194:37391)
0806 21:08:15.468018 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 21:08:15.468023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:08:15.468947 (+ 924us) server_negotiation.cc:407] Connection header received
0806 21:08:15.469017 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:08:15.469020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:08:15.469074 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:08:15.469156 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:08:15.469569 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.470365 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.471432 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.472021 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.472149 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:08:15.472217 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 21:08:15.473611 (+ 1394us) server_negotiation.cc:299] Negotiation successful
0806 21:08:15.473693 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 21:08:15.475564 7875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e183930f1eb842bf937b5f5ff49b3391" candidate_uuid: "b4cb11b97c5f46f39cac5cb710582058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643a95ad541e4e978737cfd2feaa92d6" is_pre_election: true
I20260806 21:08:15.475924 7875 raft_consensus.cc:2454] T e183930f1eb842bf937b5f5ff49b3391 P 643a95ad541e4e978737cfd2feaa92d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4cb11b97c5f46f39cac5cb710582058 in term 0.
I20260806 21:08:15.476482 7943 leader_election.cc:302] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643a95ad541e4e978737cfd2feaa92d6, b4cb11b97c5f46f39cac5cb710582058; no voters:
I20260806 21:08:15.476804 8086 raft_consensus.cc:2781] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:08:15.476904 8086 raft_consensus.cc:482] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:08:15.476948 8086 raft_consensus.cc:3037] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:08:15.478536 8060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:08:15.468428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:38305 (local address 127.27.225.195:37053)
0806 21:08:15.472889 (+ 4461us) negotiation.cc:107] Waiting for socket to connect
0806 21:08:15.472904 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:08:15.472990 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:08:15.474544 (+ 1554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:08:15.474548 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:08:15.474571 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:08:15.474651 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:08:15.474656 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.475641 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.475644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:08:15.476643 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:08:15.476652 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.478004 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.478007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:08:15.478153 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:08:15.478175 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 21:08:15.478260 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 21:08:15.478347 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4405,"thread_start_us":122,"threads_started":1}
I20260806 21:08:15.478506 8086 raft_consensus.cc:504] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643a95ad541e4e978737cfd2feaa92d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37391 } } peers { permanent_uuid: "b4cb11b97c5f46f39cac5cb710582058" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34145 } } peers { permanent_uuid: "917b25b34b5f4aa2b8e988f3fa509339" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 38305 } }
I20260806 21:08:15.478948 8076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:08:15.468829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:37053 (local address 127.27.225.193:38305)
0806 21:08:15.468959 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 21:08:15.468964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:08:15.473163 (+ 4199us) server_negotiation.cc:407] Connection header received
0806 21:08:15.473240 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:08:15.473243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:08:15.473300 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:08:15.473378 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:08:15.474794 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.475502 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.477184 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.477749 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.477892 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:08:15.477941 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 21:08:15.478743 (+ 802us) server_negotiation.cc:299] Negotiation successful
0806 21:08:15.478822 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 21:08:15.478932 8086 leader_election.cc:288] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058 [CANDIDATE]: Term 1 election: Requested vote from peers 643a95ad541e4e978737cfd2feaa92d6 (127.27.225.194:37391), 917b25b34b5f4aa2b8e988f3fa509339 (127.27.225.193:38305)
I20260806 21:08:15.479598 7875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e183930f1eb842bf937b5f5ff49b3391" candidate_uuid: "b4cb11b97c5f46f39cac5cb710582058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643a95ad541e4e978737cfd2feaa92d6"
I20260806 21:08:15.479590 7725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e183930f1eb842bf937b5f5ff49b3391" candidate_uuid: "b4cb11b97c5f46f39cac5cb710582058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "917b25b34b5f4aa2b8e988f3fa509339" is_pre_election: true
I20260806 21:08:15.479766 7875 raft_consensus.cc:3037] T e183930f1eb842bf937b5f5ff49b3391 P 643a95ad541e4e978737cfd2feaa92d6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:08:15.479943 7725 raft_consensus.cc:2454] T e183930f1eb842bf937b5f5ff49b3391 P 917b25b34b5f4aa2b8e988f3fa509339 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4cb11b97c5f46f39cac5cb710582058 in term 0.
I20260806 21:08:15.480269 7735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e183930f1eb842bf937b5f5ff49b3391" candidate_uuid: "b4cb11b97c5f46f39cac5cb710582058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "917b25b34b5f4aa2b8e988f3fa509339"
I20260806 21:08:15.480393 7735 raft_consensus.cc:3037] T e183930f1eb842bf937b5f5ff49b3391 P 917b25b34b5f4aa2b8e988f3fa509339 [term 0 FOLLOWER]: Advancing to term 1
W20260806 21:08:15.481081 8059 tablet.cc:2369] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:08:15.481101 7875 raft_consensus.cc:2454] T e183930f1eb842bf937b5f5ff49b3391 P 643a95ad541e4e978737cfd2feaa92d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4cb11b97c5f46f39cac5cb710582058 in term 1.
I20260806 21:08:15.481468 7943 leader_election.cc:302] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643a95ad541e4e978737cfd2feaa92d6, b4cb11b97c5f46f39cac5cb710582058; no voters:
I20260806 21:08:15.481536 7735 raft_consensus.cc:2454] T e183930f1eb842bf937b5f5ff49b3391 P 917b25b34b5f4aa2b8e988f3fa509339 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4cb11b97c5f46f39cac5cb710582058 in term 1.
I20260806 21:08:15.484329 8086 raft_consensus.cc:2781] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:08:15.484627 8086 raft_consensus.cc:686] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058 [term 1 LEADER]: Becoming Leader. State: Replica: b4cb11b97c5f46f39cac5cb710582058, State: Running, Role: LEADER
I20260806 21:08:15.485036 8086 consensus_queue.cc:237] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643a95ad541e4e978737cfd2feaa92d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37391 } } peers { permanent_uuid: "b4cb11b97c5f46f39cac5cb710582058" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34145 } } peers { permanent_uuid: "917b25b34b5f4aa2b8e988f3fa509339" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 38305 } }
I20260806 21:08:15.488658 7561 catalog_manager.cc:5495] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4cb11b97c5f46f39cac5cb710582058 (127.27.225.195). New cstate: current_term: 1 leader_uuid: "b4cb11b97c5f46f39cac5cb710582058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643a95ad541e4e978737cfd2feaa92d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4cb11b97c5f46f39cac5cb710582058" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "917b25b34b5f4aa2b8e988f3fa509339" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 38305 } health_report { overall_health: UNKNOWN } } }
I20260806 21:08:15.507640 8074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:08:15.501596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36364 (local address 127.27.225.195:34145)
0806 21:08:15.501731 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 21:08:15.501736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:08:15.501750 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:08:15.501956 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:08:15.501960 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:08:15.502026 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:08:15.502133 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:08:15.502447 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.503226 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.505224 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:08:15.505572 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:08:15.505685 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:08:15.506207 (+ 522us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:08:15.506801 (+ 594us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:08:15.506925 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 21:08:15.507396 (+ 471us) server_negotiation.cc:299] Negotiation successful
0806 21:08:15.507466 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 21:08:15.514977 7875 raft_consensus.cc:1261] T e183930f1eb842bf937b5f5ff49b3391 P 643a95ad541e4e978737cfd2feaa92d6 [term 1 FOLLOWER]: Refusing update from remote peer b4cb11b97c5f46f39cac5cb710582058: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:08:15.515857 7735 raft_consensus.cc:1261] T e183930f1eb842bf937b5f5ff49b3391 P 917b25b34b5f4aa2b8e988f3fa509339 [term 1 FOLLOWER]: Refusing update from remote peer b4cb11b97c5f46f39cac5cb710582058: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:08:15.516232 8086 consensus_queue.cc:1035] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643a95ad541e4e978737cfd2feaa92d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:08:15.516534 8089 consensus_queue.cc:1035] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "917b25b34b5f4aa2b8e988f3fa509339" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 38305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:08:15.528829 8092 mvcc.cc:205] Tried to move back new op lower bound from 7315662829620727808 to 7315662829509677056. Current Snapshot: MvccSnapshot[applied={T|T < 7315662829620727808}]
I20260806 21:08:15.536881 8093 mvcc.cc:205] Tried to move back new op lower bound from 7315662829620727808 to 7315662829509677056. Current Snapshot: MvccSnapshot[applied={T|T < 7315662829620727808 or (T in {7315662829620727808})}]
I20260806 21:08:15.549932 7561 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:08:15.550145 7561 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:08:15.553356 7561 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 789eb9bbc6704497b76ee2895bde2476: Sending DeleteTablet for 3 replicas of tablet e183930f1eb842bf937b5f5ff49b3391
I20260806 21:08:15.554594 7852 tablet_service.cc:1514] Processing DeleteTablet for tablet e183930f1eb842bf937b5f5ff49b3391 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:08:15 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:60648
I20260806 21:08:15.555142 7991 tablet_service.cc:1514] Processing DeleteTablet for tablet e183930f1eb842bf937b5f5ff49b3391 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:08:15 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:36350
I20260806 21:08:15.556255 7715 tablet_service.cc:1514] Processing DeleteTablet for tablet e183930f1eb842bf937b5f5ff49b3391 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:08:15 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:59902
I20260806 21:08:15.556967 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 7645
I20260806 21:08:15.562563 8102 tablet_replica.cc:330] T e183930f1eb842bf937b5f5ff49b3391 P 643a95ad541e4e978737cfd2feaa92d6: stopping tablet replica
I20260806 21:08:15.562973 8102 raft_consensus.cc:2229] T e183930f1eb842bf937b5f5ff49b3391 P 643a95ad541e4e978737cfd2feaa92d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:08:15.563375 8102 raft_consensus.cc:2258] T e183930f1eb842bf937b5f5ff49b3391 P 643a95ad541e4e978737cfd2feaa92d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:08:15.563758 8103 tablet_replica.cc:330] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058: stopping tablet replica
I20260806 21:08:15.564067 8103 raft_consensus.cc:2229] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:08:15.564667 8103 raft_consensus.cc:2258] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:08:15.568017 8103 ts_tablet_manager.cc:1905] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:08:15.569195 8102 ts_tablet_manager.cc:1905] T e183930f1eb842bf937b5f5ff49b3391 P 643a95ad541e4e978737cfd2feaa92d6: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 21:08:15.581022 7549 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.225.193:38305 (error 108)
I20260806 21:08:15.582780 8073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:08:15.582463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:38305 (local address 127.0.0.1:59904)
0806 21:08:15.582594 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 21:08:15.582681 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.193:38305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 21:08:15.582767 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 7788
W20260806 21:08:15.583359 7549 catalog_manager.cc:4584] TS 917b25b34b5f4aa2b8e988f3fa509339 (127.27.225.193:38305): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e183930f1eb842bf937b5f5ff49b3391: Network error: Client connection negotiation failed: client connection to 127.27.225.193:38305: connect: Connection refused (error 111)
I20260806 21:08:15.602689 8103 ts_tablet_manager.cc:1918] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:08:15.602865 8103 log.cc:1196] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/e183930f1eb842bf937b5f5ff49b3391
I20260806 21:08:15.603318 8103 ts_tablet_manager.cc:1939] T e183930f1eb842bf937b5f5ff49b3391 P b4cb11b97c5f46f39cac5cb710582058: Deleting consensus metadata
I20260806 21:08:15.604911 7547 catalog_manager.cc:4841] TS b4cb11b97c5f46f39cac5cb710582058 (127.27.225.195:34145): tablet e183930f1eb842bf937b5f5ff49b3391 (table test-table [id=1e3c452fd3f246fe8eee877bfde39b76]) successfully deleted
W20260806 21:08:15.611498 8073 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:08:15.606472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:37391 (local address 127.0.0.1:60660)
0806 21:08:15.606639 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 21:08:15.606654 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:08:15.606749 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:08:15.611361 (+ 4612us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.194:37391: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 21:08:15.612228 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 7926
I20260806 21:08:15.621603 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 7529
I20260806 21:08:15.633644 28551 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 21:08:15.980393 28551 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 21:08:15.980561 28551 mini_ranger.cc:66] Stopping Ranger...
I20260806 21:08:16.327067 28551 mini_ranger.cc:68] Stopped Ranger
2026-08-06 21:08:16.327 UTC [7104] LOG: received smart shutdown request
2026-08-06 21:08:16.331 UTC [7104] LOG: background worker "logical replication launcher" (PID 7113) exited with exit code 1
2026-08-06 21:08:16.332 UTC [7108] LOG: shutting down
2026-08-06 21:08:16.540 UTC [7104] LOG: database system is shut down
2026-08-06T21:08:16Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50628 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:08:16 dist-test-slave-69rg krb5kdc[8114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:08:16 dist-test-slave-69rg krb5kdc[8114](info): set up 2 sockets
Aug 06 21:08:16 dist-test-slave-69rg krb5kdc[8114](info): commencing operation
krb5kdc: starting...
W20260806 21:08:17.767694 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:08:17 dist-test-slave-69rg krb5kdc[8114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:08:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:08:17Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.27.225.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.27.225.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.27.225.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/rangeradmin_127.27.225.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.27.225.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/rangeradmin_127.27.225.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.27.225.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.27.225.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.27.225.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/rangerlookup_127.27.225.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.27.225.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/rangerlookup_127.27.225.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.27.225.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.27.225.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.27.225.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/HTTP_127.27.225.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.27.225.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/HTTP_127.27.225.212@KRBTEST.COM.keytab.
I20260806 21:08:17.964651 28551 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDiceBW/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/postgres -l logfile start
2026-08-06 21:08:20.808 UTC [8145] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 21:08:20.809 UTC [8145] LOG: listening on IPv4 address "127.27.225.212", port 44817
2026-08-06 21:08:20.813 UTC [8145] LOG: listening on Unix socket "/tmp/.s.PGSQL.44817"
2026-08-06 21:08:20.838 UTC [8148] LOG: database system was shut down at 2026-08-06 21:08:19 UTC
2026-08-06 21:08:20.843 UTC [8145] LOG: database system is ready to accept connections
I20260806 21:08:21.882704 28551 mini_postgres.cc:96] Postgres bound to 44817
2026-08-06 21:08:21.891 UTC [8156] FATAL: database "slave" does not exist
127.27.225.212:44817 - accepting connections
I20260806 21:08:21.891772 28551 mini_ranger.cc:161] Starting Ranger...
I20260806 21:08:21.911589 28551 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 21:08:22.685115 28551 mini_ranger.cc:87] Created ranger Postgres database
I20260806 21:08:22.685200 28551 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ranger-admin
2026-08-06 21:08:22,995 [I] DB FLAVOR :POSTGRES
2026-08-06 21:08:22,995 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 21:08:22,995 [I] Checking connection..
2026-08-06 21:08:22,995 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:44817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 21:08:23,273 [I] Checking connection passed.
2026-08-06 21:08:23,274 [I] --------- Verifying version history table ---------
2026-08-06 21:08:23,274 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:44817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 21:08:23,575 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 21:08:23,575 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:44817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 21:08:23,846 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 21:08:23,846 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 21:08:23,846 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:44817/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 21:08:24.106 UTC [8223] WARNING: there is no transaction in progress
2026-08-06 21:08:24,113 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 21:08:24,114 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:44817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 21:08:24,364 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 21:08:24,364 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 21:08:24,364 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:44817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 21:08:24,632 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:44817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 21:08:24,890 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:44817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-69rg.c.gcp-upstream.internal','N') ;"
2026-08-06 21:08:25,147 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:44817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 21:08:25,396 [I] Table x_portal_user does not exist in database ranger
2026-08-06 21:08:25,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:44817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 21:08:25,649 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 21:08:25,649 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:44817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 21:08:25,885 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:44817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 21:08:26,138 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 21:08:26,138 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:44817/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 21:08:27.178 UTC [8332] WARNING: there is no transaction in progress
2026-08-06 21:08:27.196 UTC [8332] WARNING: there is no transaction in progress
2026-08-06 21:08:27.213 UTC [8332] WARNING: there is no transaction in progress
2026-08-06 21:08:27.230 UTC [8332] WARNING: there is no transaction in progress
2026-08-06 21:08:27.249 UTC [8332] WARNING: there is no transaction in progress
2026-08-06 21:08:27.268 UTC [8332] WARNING: there is no transaction in progress
2026-08-06 21:08:27.329 UTC [8332] WARNING: there is no transaction in progress
2026-08-06 21:08:27.337 UTC [8332] WARNING: there is no transaction in progress
2026-08-06 21:08:27.355 UTC [8332] WARNING: there is no transaction in progress
2026-08-06 21:08:27.366 UTC [8332] WARNING: there is no transaction in progress
2026-08-06 21:08:27.378 UTC [8332] WARNING: there is no transaction in progress
2026-08-06 21:08:27.393 UTC [8332] WARNING: there is no transaction in progress
2026-08-06 21:08:27.405 UTC [8332] WARNING: there is no transaction in progress
2026-08-06 21:08:27.412 UTC [8332] WARNING: there is no transaction in progress
2026-08-06 21:08:27.420 UTC [8332] WARNING: there is no transaction in progress
2026-08-06 21:08:27.428 UTC [8332] WARNING: there is no transaction in progress
2026-08-06 21:08:28,218 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 21:08:28,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:44817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 21:08:28,470 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 21:08:28,471 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:44817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 21:08:28,744 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 21:08:28,744 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:44817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 21:08:28,984 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 21:08:28,984 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:44817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 21:08:29,251 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:44817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 21:08:29,509 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:44817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-69rg.c.gcp-upstream.internal';"
2026-08-06 21:08:29,796 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 21:08:29,796 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:44817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 21:08:30,070 [I] DB_PATCHES have already been applied
I20260806 21:08:30.076089 28551 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDiceBW/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 21:08:30.076176 28551 mini_ranger.cc:193] Using host: 127.27.225.212
I20260806 21:08:30.080981 28551 mini_ranger.cc:227] Ranger admin URL: http://127.27.225.212:46637
Aug 06, 2026 9:08:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 9:08:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 9:08:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 9:08:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 9:08:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 9:08:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 9:08:31 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 21:08:31.256279 28551 mini_ranger.cc:160] Time spent starting Ranger: real 9.365s user 0.005s sys 0.009s
Aug 06, 2026 9:08:51 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 9:08:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 06, 2026 9:08:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 9:08:51 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 9:08:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 9:08:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 9:08:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 9:08:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 9:08:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 9:08:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 9:08:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 9:08:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 9:08:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 9:08:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 9:08:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 9:08:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 9:08:52 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 9:08:52 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 21:08:53.570029 28551 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.27.225.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.27.225.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.27.225.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/rangerkms_127.27.225.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.27.225.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/rangerkms_127.27.225.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.27.225.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.27.225.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.27.225.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/HTTP_127.27.225.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.27.225.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/HTTP_127.27.225.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 21:08:53 dist-test-slave-69rg krb5kdc[8114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050533, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 21:08:53.668391 28551 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 21:08:53.680051 28551 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 21:08:54.746238 28551 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 21:08:54.746317 28551 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ranger-kms
I20260806 21:08:54.746332 28551 mini_ranger_kms.cc:227] Using postgres at 127.27.225.212:44817
2026-08-06 21:08:54,941 [I] DB FLAVOR :POSTGRES
2026-08-06 21:08:54,942 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 21:08:54,942 [I] Checking connection
2026-08-06 21:08:54,942 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:44817/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 21:08:55,185 [I] connection success
2026-08-06 21:08:55,185 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 21:08:55,185 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 21:08:55,185 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:44817/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 21:08:55,427 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 21:08:55,427 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 21:08:55,427 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 21:08:55,427 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.225.212:44817/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 21:08:55,736 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 21:08:56.246567 28551 mini_ranger_kms.cc:310] Created kms service
I20260806 21:08:56.422289 28551 mini_ranger_kms.cc:326] Created kudu user
I20260806 21:08:56.469508 28551 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 21:08:56 dist-test-slave-69rg krb5kdc[8114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050533, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.27.225.212@KRBTEST.COM
I20260806 21:08:56.914057 28551 mini_ranger_kms.cc:384] Added ranger policy
I20260806 21:08:56.914224 28551 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDiceBW/build/debug/bin/postgresql.jar:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDiceBW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 21:08:56.914247 28551 mini_ranger_kms.cc:240] Using host: 127.27.225.212
I20260806 21:08:56.921170 28551 mini_ranger_kms.cc:276] Ranger KMS PID: 8516
I20260806 21:08:56.921268 28551 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.27.225.212:54835
Aug 06, 2026 9:08:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 9:08:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 9:08:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 9:08:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 9:08:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 9:08:58 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 21:08:59.188369 28551 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.520s user 0.014s sys 0.167s
I20260806 21:08:59.188478 28551 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 21:08:59.188532 28551 mini_ranger_kms.cc:401] 127.27.225.212:54835/kms/v1/keys
Aug 06 21:08:59 dist-test-slave-69rg krb5kdc[8114](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786050539, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.225.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 21:08:59 dist-test-slave-69rg krb5kdc[8114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050539, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.225.212@KRBTEST.COM for HTTP/127.27.225.212@KRBTEST.COM
Aug 06 21:09:00 dist-test-slave-69rg krb5kdc[8114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050539, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.225.212@KRBTEST.COM for HTTP/127.27.225.212@KRBTEST.COM
Aug 06 21:09:02 dist-test-slave-69rg krb5kdc[8114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:09:02.477087 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:41611
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:43939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:41611
--ranger_config_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.225.212:54835/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:09:02.593233 8568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:02.593503 8568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:02.593542 8568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:02.597617 8568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:09:02.597690 8568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:02.597715 8568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:02.597733 8568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:09:02.597748 8568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:09:02.603504 8568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:43939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/wal
--ranger_kms_url=127.27.225.212:54835/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:41611
--ranger_config_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:41611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:02.604655 8568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:02.605727 8568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:02.614817 8576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:02.614821 8573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:02.614821 8574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:02.614859 8568 server_base.cc:1034] running on GCE node
I20260806 21:09:02.615622 8568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:02.616698 8568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:02.617890 8568 hybrid_clock.cc:648] HybridClock initialized: now 1786050542617862 us; error 42 us; skew 500 ppm
Aug 06 21:09:02 dist-test-slave-69rg krb5kdc[8114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050542, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:02.620781 8568 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:09:02.621706 8568 webserver.cc:460] Webserver started at http://127.27.225.254:43461/ using document root <none> and password file <none>
I20260806 21:09:02.622335 8568 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:02.622393 8568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:02.622608 8568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 21:09:02 dist-test-slave-69rg krb5kdc[8114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050542, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for HTTP/127.27.225.212@KRBTEST.COM
I20260806 21:09:02.688306 8568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "44d7afc671e54c36935efafee3b9ca62"
format_stamp: "Formatted at 2026-08-06 21:09:02 on dist-test-slave-69rg"
server_key: "b4cf9d7f77c729ec6dcbc7576639f307"
server_key_iv: "e405f165984b2dbb778e7d8aa20f0d3d"
server_key_version: "kuduclusterkey@0"
I20260806 21:09:02.688943 8568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "44d7afc671e54c36935efafee3b9ca62"
format_stamp: "Formatted at 2026-08-06 21:09:02 on dist-test-slave-69rg"
server_key: "b4cf9d7f77c729ec6dcbc7576639f307"
server_key_iv: "e405f165984b2dbb778e7d8aa20f0d3d"
server_key_version: "kuduclusterkey@0"
I20260806 21:09:02.693248 8568 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:09:02.714074 8583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:02.715554 8568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:09:02.715715 8568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "44d7afc671e54c36935efafee3b9ca62"
format_stamp: "Formatted at 2026-08-06 21:09:02 on dist-test-slave-69rg"
server_key: "b4cf9d7f77c729ec6dcbc7576639f307"
server_key_iv: "e405f165984b2dbb778e7d8aa20f0d3d"
server_key_version: "kuduclusterkey@0"
I20260806 21:09:02.715869 8568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:02.729188 8568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:02.732847 8568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:02.733076 8568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:02.741731 8568 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:41611
I20260806 21:09:02.741801 8635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:41611 every 8 connection(s)
I20260806 21:09:02.742894 8568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:09:02.745704 8636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:09:02.747145 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 8568
I20260806 21:09:02.747277 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:09:02.747573 28551 external_mini_cluster.cc:1428] Setting key 9ee5b7555ded03c647e1ed7d4c13d92d
I20260806 21:09:02.752750 8636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62: Bootstrap starting.
I20260806 21:09:02.755270 8636 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62: Neither blocks nor log segments found. Creating new log.
I20260806 21:09:02.756184 8636 log.cc:824] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62: Log is configured to *not* fsync() on all Append() calls
I20260806 21:09:02.758420 8636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62: No bootstrap required, opened a new log
Aug 06 21:09:02 dist-test-slave-69rg krb5kdc[8114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:09:02.762950 8636 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44d7afc671e54c36935efafee3b9ca62" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41611 } }
I20260806 21:09:02.763289 8636 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:09:02.763375 8636 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44d7afc671e54c36935efafee3b9ca62, State: Initialized, Role: FOLLOWER
I20260806 21:09:02.763990 8636 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44d7afc671e54c36935efafee3b9ca62" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41611 } }
I20260806 21:09:02.764158 8636 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:09:02.764204 8636 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:09:02.764307 8636 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:09:02.765592 8636 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44d7afc671e54c36935efafee3b9ca62" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41611 } }
I20260806 21:09:02.766009 8641 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:09:02.766000 8636 leader_election.cc:302] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44d7afc671e54c36935efafee3b9ca62; no voters:
I20260806 21:09:02.766165 8641 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44d7afc671e54c36935efafee3b9ca62" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41611 } }
I20260806 21:09:02.766417 8639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:02.749119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38684 (local address 127.27.225.254:41611)
0806 21:09:02.749603 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:02.749623 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:02.749667 (+ 44us) server_negotiation.cc:407] Connection header received
0806 21:09:02.750559 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:02.750584 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:02.751016 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:02.751509 (+ 493us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:02.752412 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:02.753879 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:02.755151 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:02.755889 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:02.756064 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:02.760036 (+ 3972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:02.760050 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:02.760070 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:02.760131 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:02.762378 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:02.762968 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:02.762977 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:02.762982 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:02.763119 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:02.763477 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:02.763481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:02.763483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:02.764068 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:02.764277 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:02.764779 (+ 502us) server_negotiation.cc:299] Negotiation successful
0806 21:09:02.765296 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":161,"threads_started":1}
I20260806 21:09:02.766470 8636 leader_election.cc:288] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:09:02.766947 8642 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:09:02.766979 8641 leader_election.cc:302] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44d7afc671e54c36935efafee3b9ca62; no voters:
I20260806 21:09:02.767251 8642 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [term 1 LEADER]: Becoming Leader. State: Replica: 44d7afc671e54c36935efafee3b9ca62, State: Running, Role: LEADER
I20260806 21:09:02.767637 8642 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44d7afc671e54c36935efafee3b9ca62" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41611 } }
I20260806 21:09:02.767963 8636 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:09:02.768833 8642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44d7afc671e54c36935efafee3b9ca62. Latest consensus state: current_term: 1 leader_uuid: "44d7afc671e54c36935efafee3b9ca62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44d7afc671e54c36935efafee3b9ca62" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41611 } } }
I20260806 21:09:02.768944 8642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [sys.catalog]: This master's current role is: LEADER
I20260806 21:09:02.767251 8641 leader_election.cc:288] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers
I20260806 21:09:02.769482 8644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44d7afc671e54c36935efafee3b9ca62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44d7afc671e54c36935efafee3b9ca62" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41611 } } }
I20260806 21:09:02.769613 8644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [sys.catalog]: This master's current role is: LEADER
I20260806 21:09:02.770427 8636 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 21:09:02.770606 8652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:09:02.776463 8652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:09:02.784811 8652 catalog_manager.cc:1347] Generated new cluster ID: 82f1d978d5db47ef8278de2ddc56afa7
I20260806 21:09:02.784907 8652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:09:02.810544 8652 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:09:02.812565 8652 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:09:02.825194 8652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62: Generated new TSK 0
I20260806 21:09:02.826145 8652 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.225.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
I20260806 21:09:03.490320 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.225.212:54835/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:41611
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:43939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:09:03.630754 8684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:03.631134 8684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:03.631206 8684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:03.636897 8684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:03.636998 8684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:03.637090 8684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:09:03.643553 8684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:43939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.27.225.212:54835/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:41611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:03.645066 8684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:03.646518 8684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:03.658216 8692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:03.658443 8691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:03.658547 8694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:03.658800 8684 server_base.cc:1034] running on GCE node
I20260806 21:09:03.659271 8684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:03.660017 8684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:03.661238 8684 hybrid_clock.cc:648] HybridClock initialized: now 1786050543661164 us; error 93 us; skew 500 ppm
Aug 06 21:09:03 dist-test-slave-69rg krb5kdc[8114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050543, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:03.665566 8684 init.cc:377] Logged in from keytab as kudu/127.27.225.193@KRBTEST.COM (short username kudu)
I20260806 21:09:03.666900 8684 webserver.cc:460] Webserver started at http://127.27.225.193:45571/ using document root <none> and password file <none>
I20260806 21:09:03.667686 8684 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:03.667790 8684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:03.668048 8684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 21:09:03 dist-test-slave-69rg krb5kdc[8114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050543, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for HTTP/127.27.225.212@KRBTEST.COM
I20260806 21:09:03.687712 8684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "634d8826fc8048b8a3732d086431f50e"
format_stamp: "Formatted at 2026-08-06 21:09:03 on dist-test-slave-69rg"
server_key: "c23e5d01d013c50e5ec903e636c59f47"
server_key_iv: "af3ee465e7aff3132ee8816b9b4ab7ac"
server_key_version: "kuduclusterkey@0"
I20260806 21:09:03.688560 8684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "634d8826fc8048b8a3732d086431f50e"
format_stamp: "Formatted at 2026-08-06 21:09:03 on dist-test-slave-69rg"
server_key: "c23e5d01d013c50e5ec903e636c59f47"
server_key_iv: "af3ee465e7aff3132ee8816b9b4ab7ac"
server_key_version: "kuduclusterkey@0"
I20260806 21:09:03.694013 8684 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 21:09:03.722347 8702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:03.724274 8684 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.003s sys 0.001s
I20260806 21:09:03.724458 8684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "634d8826fc8048b8a3732d086431f50e"
format_stamp: "Formatted at 2026-08-06 21:09:03 on dist-test-slave-69rg"
server_key: "c23e5d01d013c50e5ec903e636c59f47"
server_key_iv: "af3ee465e7aff3132ee8816b9b4ab7ac"
server_key_version: "kuduclusterkey@0"
I20260806 21:09:03.724627 8684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:03.754128 8684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:03.758213 8684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:03.758431 8684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:03.759133 8684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:09:03.760296 8684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:09:03.760367 8684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:03.760433 8684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:09:03.760460 8684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:03.772310 8684 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:33225
I20260806 21:09:03.773512 8684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:09:03.774798 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 8684
I20260806 21:09:03.774922 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:09:03.775285 28551 external_mini_cluster.cc:1428] Setting key e814772bfa39ef2474e329cc1cefb56d
I20260806 21:09:03.781054 8821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:33225 every 8 connection(s)
WARNING: no policy specified for kudu/127.27.225.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.194@KRBTEST.COM" created.
Aug 06 21:09:03 dist-test-slave-69rg krb5kdc[8114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050543, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:09:03.802294 8826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:03.784977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:54027 (local address 127.27.225.254:41611)
0806 21:09:03.787186 (+ 2209us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:03.787195 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:03.787214 (+ 19us) server_negotiation.cc:407] Connection header received
0806 21:09:03.787260 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:03.787263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:03.787317 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:03.787415 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:03.789071 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:03.789990 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:03.791927 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:03.792386 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:03.792503 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:03.796764 (+ 4261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:03.796768 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:03.796771 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:03.796806 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:03.799007 (+ 2201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:03.799787 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:03.799791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:03.799793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:03.799897 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:03.800373 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:03.800376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:03.800378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:03.800642 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:03.800763 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:03.801941 (+ 1178us) server_negotiation.cc:299] Negotiation successful
0806 21:09:03.802079 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2115,"thread_start_us":112,"threads_started":1}
I20260806 21:09:03.803762 8825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:03.783796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41611 (local address 127.27.225.193:54027)
0806 21:09:03.785508 (+ 1712us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:03.785568 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:03.786720 (+ 1152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:03.787745 (+ 1025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:03.787755 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:03.788138 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:03.788863 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:03.788887 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:03.790149 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:03.790153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:03.791755 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:03.791769 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:03.792733 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:03.792738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:03.792969 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:03.793739 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:03.793763 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:03.796391 (+ 2628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:03.799177 (+ 2786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:03.799187 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:03.799208 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:03.799636 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:03.800021 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:03.800026 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:03.800028 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:03.800247 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:03.800825 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:03.800833 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:03.801205 (+ 372us) client_negotiation.cc:769] Sending connection context
0806 21:09:03.801430 (+ 225us) client_negotiation.cc:240] Negotiation successful
0806 21:09:03.802564 (+ 1134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1478,"thread_start_us":154,"threads_started":1}
I20260806 21:09:03.805251 8823 heartbeater.cc:347] Connected to a master server at 127.27.225.254:41611
I20260806 21:09:03.805540 8823 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:03.806209 8823 heartbeater.cc:511] Master 127.27.225.254:41611 requested a full tablet report, sending...
I20260806 21:09:03.808336 8600 ts_manager.cc:194] Registered new tserver with Master: 634d8826fc8048b8a3732d086431f50e (127.27.225.193:33225)
I20260806 21:09:03.810055 8600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.193@KRBTEST.COM'} at 127.27.225.193:54027
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
I20260806 21:09:03.816253 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.225.212:54835/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:41611
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:43939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 21:09:03 dist-test-slave-69rg krb5kdc[8114](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786050543, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 21:09:03.962960 8828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:03.963373 8828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:03.963446 8828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:03.969569 8828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:03.969666 8828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:03.969758 8828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:09:03.978168 8828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:43939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.27.225.212:54835/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:41611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:03.979664 8828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:03.981009 8828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:09:03.994094 8828 server_base.cc:1034] running on GCE node
W20260806 21:09:03.994271 8835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:03.994601 8834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:03.994800 8837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:03.995361 8828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:03.996209 8828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:03.997524 8828 hybrid_clock.cc:648] HybridClock initialized: now 1786050543997450 us; error 91 us; skew 500 ppm
Aug 06 21:09:03 dist-test-slave-69rg krb5kdc[8114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050543, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:04.001389 8828 init.cc:377] Logged in from keytab as kudu/127.27.225.194@KRBTEST.COM (short username kudu)
I20260806 21:09:04.002612 8828 webserver.cc:460] Webserver started at http://127.27.225.194:33593/ using document root <none> and password file <none>
I20260806 21:09:04.003322 8828 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:04.003413 8828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:04.003633 8828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 21:09:04 dist-test-slave-69rg krb5kdc[8114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050543, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for HTTP/127.27.225.212@KRBTEST.COM
I20260806 21:09:04.025152 8828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "d00e7abc8a2f43cba94db4596c11529a"
format_stamp: "Formatted at 2026-08-06 21:09:04 on dist-test-slave-69rg"
server_key: "46c6535d986f4cb55244954309a40bca"
server_key_iv: "35e65d812c3824e7842b9344bfd068c4"
server_key_version: "kuduclusterkey@0"
I20260806 21:09:04.026144 8828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "d00e7abc8a2f43cba94db4596c11529a"
format_stamp: "Formatted at 2026-08-06 21:09:04 on dist-test-slave-69rg"
server_key: "46c6535d986f4cb55244954309a40bca"
server_key_iv: "35e65d812c3824e7842b9344bfd068c4"
server_key_version: "kuduclusterkey@0"
I20260806 21:09:04.037079 8828 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.001s sys 0.003s
I20260806 21:09:04.062217 8844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:04.064095 8828 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.001s sys 0.000s
I20260806 21:09:04.064261 8828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "d00e7abc8a2f43cba94db4596c11529a"
format_stamp: "Formatted at 2026-08-06 21:09:04 on dist-test-slave-69rg"
server_key: "46c6535d986f4cb55244954309a40bca"
server_key_iv: "35e65d812c3824e7842b9344bfd068c4"
server_key_version: "kuduclusterkey@0"
I20260806 21:09:04.064419 8828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:04.099272 8828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:04.102672 8828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:04.102918 8828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:04.103581 8828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:09:04.104522 8828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:09:04.104599 8828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:04.104671 8828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:09:04.104708 8828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:04.121582 8828 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:42679
I20260806 21:09:04.123091 8828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:09:04.124006 8962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:42679 every 8 connection(s)
I20260806 21:09:04.128279 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 8828
I20260806 21:09:04.128427 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:09:04.128839 28551 external_mini_cluster.cc:1428] Setting key 6cec7977b245669f786ebf69238e21e0
Aug 06 21:09:04 dist-test-slave-69rg krb5kdc[8114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050543, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:09:04.144882 8826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:04.127090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:39469 (local address 127.27.225.254:41611)
0806 21:09:04.127257 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:04.127262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:04.129340 (+ 2078us) server_negotiation.cc:407] Connection header received
0806 21:09:04.131202 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:04.131206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:04.131266 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:04.131323 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:04.133134 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:04.134065 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:04.135878 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:04.136275 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:04.136372 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:04.139930 (+ 3558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:04.139935 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:04.139938 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:04.139971 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:04.141988 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:04.142731 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:04.142735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:04.142737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:04.142853 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:04.143311 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:04.143314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:04.143316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:04.143566 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:04.143644 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:04.144420 (+ 776us) server_negotiation.cc:299] Negotiation successful
0806 21:09:04.144579 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 21:09:04.147289 8965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:04.127616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41611 (local address 127.27.225.194:39469)
0806 21:09:04.129126 (+ 1510us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:04.129193 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:04.130818 (+ 1625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:04.131830 (+ 1012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:04.131841 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:04.132234 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:04.132947 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:04.132967 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:04.134234 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:04.134238 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:04.135706 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:04.135718 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:04.136621 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:04.136626 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:04.136874 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:04.137655 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:04.137678 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:04.139665 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:04.142153 (+ 2488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:04.142160 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:04.142180 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:04.142592 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:04.142977 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:04.142980 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:04.142982 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:04.143189 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:04.143732 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:04.143739 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:04.144118 (+ 379us) client_negotiation.cc:769] Sending connection context
0806 21:09:04.145534 (+ 1416us) client_negotiation.cc:240] Negotiation successful
0806 21:09:04.146011 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1235,"thread_start_us":192,"threads_started":1}
I20260806 21:09:04.151794 8963 heartbeater.cc:347] Connected to a master server at 127.27.225.254:41611
I20260806 21:09:04.152086 8963 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:04.152738 8963 heartbeater.cc:511] Master 127.27.225.254:41611 requested a full tablet report, sending...
I20260806 21:09:04.154413 8600 ts_manager.cc:194] Registered new tserver with Master: d00e7abc8a2f43cba94db4596c11529a (127.27.225.194:42679)
I20260806 21:09:04.155241 8600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.194@KRBTEST.COM'} at 127.27.225.194:39469
WARNING: no policy specified for kudu/127.27.225.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
I20260806 21:09:04.219558 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.225.212:54835/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:41611
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:43939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 21:09:04 dist-test-slave-69rg krb5kdc[8114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050543, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for HTTP/127.27.225.212@KRBTEST.COM
W20260806 21:09:04.411127 8968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:04.411540 8968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:04.411624 8968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:04.418078 8968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:04.418182 8968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:04.418292 8968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:09:04.426581 8968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:43939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.27.225.212:54835/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:41611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:04.428037 8968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:04.429378 8968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:04.442063 8976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:04.442585 8974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:04.443459 8968 server_base.cc:1034] running on GCE node
W20260806 21:09:04.444446 8973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:04.445030 8968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:04.445885 8968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:04.447052 8968 hybrid_clock.cc:648] HybridClock initialized: now 1786050544446981 us; error 86 us; skew 500 ppm
Aug 06 21:09:04 dist-test-slave-69rg krb5kdc[8114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050544, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:04.450992 8968 init.cc:377] Logged in from keytab as kudu/127.27.225.195@KRBTEST.COM (short username kudu)
I20260806 21:09:04.452525 8968 webserver.cc:460] Webserver started at http://127.27.225.195:44989/ using document root <none> and password file <none>
I20260806 21:09:04.453327 8968 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:04.453482 8968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:04.453789 8968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 21:09:04 dist-test-slave-69rg krb5kdc[8114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050544, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for HTTP/127.27.225.212@KRBTEST.COM
I20260806 21:09:04.483191 8968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "5cd793a3fcb74f1db65b603d79f71faf"
format_stamp: "Formatted at 2026-08-06 21:09:04 on dist-test-slave-69rg"
server_key: "3fb07b5b9a64454d8386a843bcaef2f7"
server_key_iv: "8b4e9d37365156b88ee6591950cd63fd"
server_key_version: "kuduclusterkey@0"
I20260806 21:09:04.491767 8968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "5cd793a3fcb74f1db65b603d79f71faf"
format_stamp: "Formatted at 2026-08-06 21:09:04 on dist-test-slave-69rg"
server_key: "3fb07b5b9a64454d8386a843bcaef2f7"
server_key_iv: "8b4e9d37365156b88ee6591950cd63fd"
server_key_version: "kuduclusterkey@0"
I20260806 21:09:04.503731 8968 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.004s sys 0.000s
I20260806 21:09:04.527808 8983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:04.529666 8968 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260806 21:09:04.529999 8968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "5cd793a3fcb74f1db65b603d79f71faf"
format_stamp: "Formatted at 2026-08-06 21:09:04 on dist-test-slave-69rg"
server_key: "3fb07b5b9a64454d8386a843bcaef2f7"
server_key_iv: "8b4e9d37365156b88ee6591950cd63fd"
server_key_version: "kuduclusterkey@0"
I20260806 21:09:04.530185 8968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:04.550786 8968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:04.554854 8968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:04.555087 8968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:04.556062 8968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:09:04.557493 8968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:09:04.557578 8968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:04.557642 8968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:09:04.557687 8968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:04.571640 8968 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:44015
I20260806 21:09:04.572908 8968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:09:04.574887 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 8968
I20260806 21:09:04.575014 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:09:04.575338 28551 external_mini_cluster.cc:1428] Setting key 159a5171b04e6f67a9ac82699684d8dd
I20260806 21:09:04.574342 9099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:44015 every 8 connection(s)
Aug 06 21:09:04 dist-test-slave-69rg krb5kdc[8114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050544, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:09:04.599975 8826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:04.581392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:33979 (local address 127.27.225.254:41611)
0806 21:09:04.581525 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:04.581529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:04.582063 (+ 534us) server_negotiation.cc:407] Connection header received
0806 21:09:04.583406 (+ 1343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:04.583410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:04.583473 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:04.583531 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:04.585384 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:04.586270 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:04.589991 (+ 3721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:04.590368 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:04.590461 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:04.594854 (+ 4393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:04.594860 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:04.594864 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:04.594900 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:04.596984 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:04.597801 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:04.597805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:04.597807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:04.597941 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:04.598448 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:04.598452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:04.598453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:04.598714 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:04.598804 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:04.599609 (+ 805us) server_negotiation.cc:299] Negotiation successful
0806 21:09:04.599785 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 21:09:04.603740 9102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:04.579736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41611 (local address 127.27.225.195:33979)
0806 21:09:04.581885 (+ 2149us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:04.581950 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:04.583120 (+ 1170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:04.583997 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:04.584008 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:04.584444 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:04.585178 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:04.585199 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:04.586457 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:04.586460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:04.589796 (+ 3336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:04.589854 (+ 58us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:04.590696 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:04.590700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:04.590935 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:04.591717 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:04.591742 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:04.594518 (+ 2776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:04.597200 (+ 2682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:04.597209 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:04.597231 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:04.597652 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:04.598098 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:04.598101 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:04.598103 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:04.598322 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:04.598874 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:04.598882 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:04.599285 (+ 403us) client_negotiation.cc:769] Sending connection context
0806 21:09:04.602103 (+ 2818us) client_negotiation.cc:240] Negotiation successful
0806 21:09:04.602550 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1885,"thread_start_us":190,"threads_started":1}
I20260806 21:09:04.605058 9100 heartbeater.cc:347] Connected to a master server at 127.27.225.254:41611
I20260806 21:09:04.605335 9100 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:04.605998 9100 heartbeater.cc:511] Master 127.27.225.254:41611 requested a full tablet report, sending...
I20260806 21:09:04.607174 8600 ts_manager.cc:194] Registered new tserver with Master: 5cd793a3fcb74f1db65b603d79f71faf (127.27.225.195:44015)
I20260806 21:09:04.607980 8600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.195@KRBTEST.COM'} at 127.27.225.195:33979
I20260806 21:09:04.613888 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:09:04.614845 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 8684
I20260806 21:09:04.632510 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 8828
I20260806 21:09:04.642989 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 8968
I20260806 21:09:04.653600 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 8568
I20260806 21:09:04.662978 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:41611
--webserver_interface=127.27.225.254
--webserver_port=43461
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:43939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:41611
--ranger_config_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.225.212:54835/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.27.225.212:54835/kms with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:09:04.799340 9103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:04.799652 9103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:04.799695 9103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:04.804112 9103 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:09:04.804194 9103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:04.804219 9103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:04.804237 9103 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:09:04.804252 9103 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:09:04.810195 9103 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:43939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.27.225.212:54835/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:41611
--ranger_config_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:41611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=43461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:04.811450 9103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:04.812583 9103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:04.823549 9108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:04.823586 9109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:04.823661 9111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:04.823750 9103 server_base.cc:1034] running on GCE node
I20260806 21:09:04.824807 9103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:04.826051 9103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:04.827271 9103 hybrid_clock.cc:648] HybridClock initialized: now 1786050544827240 us; error 48 us; skew 500 ppm
Aug 06 21:09:04 dist-test-slave-69rg krb5kdc[8114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050544, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:04.830328 9103 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:09:04.831419 9103 webserver.cc:460] Webserver started at http://127.27.225.254:43461/ using document root <none> and password file <none>
I20260806 21:09:04.831992 9103 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:04.832063 9103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 21:09:04 dist-test-slave-69rg krb5kdc[8114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050544, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for HTTP/127.27.225.212@KRBTEST.COM
I20260806 21:09:04.926555 9103 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:09:04.929021 9119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:04.930620 9103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:09:04.930845 9103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "44d7afc671e54c36935efafee3b9ca62"
format_stamp: "Formatted at 2026-08-06 21:09:02 on dist-test-slave-69rg"
server_key: "b4cf9d7f77c729ec6dcbc7576639f307"
server_key_iv: "e405f165984b2dbb778e7d8aa20f0d3d"
server_key_version: "kuduclusterkey@0"
I20260806 21:09:04.931473 9103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:04.965144 9103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:04.968431 9103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:04.968786 9103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:04.977727 9103 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:41611
I20260806 21:09:04.977764 9174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:41611 every 8 connection(s)
I20260806 21:09:04.979099 9103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:09:04.983909 9175 sys_catalog.cc:263] Verifying existing consensus state
I20260806 21:09:04.984414 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 9103
I20260806 21:09:04.984894 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.225.212:54835/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:33225
--local_ip_for_outbound_sockets=127.27.225.193
--tserver_master_addrs=127.27.225.254:41611
--webserver_port=45571
--webserver_interface=127.27.225.193
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:43939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.27.225.212:54835/kms with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:09:04.985965 9175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62: Bootstrap starting.
I20260806 21:09:05.005925 9175 log.cc:824] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62: Log is configured to *not* fsync() on all Append() calls
I20260806 21:09:05.015452 9175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 21:09:05.015983 9175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62: Bootstrap complete.
I20260806 21:09:05.023937 9175 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44d7afc671e54c36935efafee3b9ca62" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41611 } }
I20260806 21:09:05.024475 9175 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44d7afc671e54c36935efafee3b9ca62, State: Initialized, Role: FOLLOWER
I20260806 21:09:05.025583 9175 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44d7afc671e54c36935efafee3b9ca62" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41611 } }
I20260806 21:09:05.025913 9175 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:09:05.026024 9175 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:09:05.026163 9175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [term 1 FOLLOWER]: Advancing to term 2
I20260806 21:09:05.028807 9175 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44d7afc671e54c36935efafee3b9ca62" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41611 } }
I20260806 21:09:05.029296 9175 leader_election.cc:302] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44d7afc671e54c36935efafee3b9ca62; no voters:
I20260806 21:09:05.029706 9175 leader_election.cc:288] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 21:09:05.029837 9179 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 21:09:05.030061 9179 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [term 2 LEADER]: Becoming Leader. State: Replica: 44d7afc671e54c36935efafee3b9ca62, State: Running, Role: LEADER
I20260806 21:09:05.030404 9179 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44d7afc671e54c36935efafee3b9ca62" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41611 } }
I20260806 21:09:05.031131 9175 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:09:05.031836 9180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "44d7afc671e54c36935efafee3b9ca62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44d7afc671e54c36935efafee3b9ca62" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41611 } } }
I20260806 21:09:05.031975 9180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [sys.catalog]: This master's current role is: LEADER
I20260806 21:09:05.032442 9181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44d7afc671e54c36935efafee3b9ca62. Latest consensus state: current_term: 2 leader_uuid: "44d7afc671e54c36935efafee3b9ca62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44d7afc671e54c36935efafee3b9ca62" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41611 } } }
I20260806 21:09:05.032593 9181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62 [sys.catalog]: This master's current role is: LEADER
I20260806 21:09:05.032920 9186 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:09:05.037307 9186 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:09:05.037985 9175 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 21:09:05.038261 9186 catalog_manager.cc:1259] Loaded cluster ID: 82f1d978d5db47ef8278de2ddc56afa7
I20260806 21:09:05.038378 9186 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:09:05.041221 9186 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:09:05.041970 9186 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62: Loaded TSK: 0
I20260806 21:09:05.043205 9186 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 21:09:05.135814 9177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:05.136174 9177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:05.136264 9177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:05.141876 9177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:05.141965 9177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:05.142066 9177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:09:05.149204 9177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:43939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.27.225.212:54835/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:33225
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=45571
--tserver_master_addrs=127.27.225.254:41611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:05.150540 9177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:05.151763 9177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:05.166846 9214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:05.166564 9213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:05.166447 9216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:05.167411 9177 server_base.cc:1034] running on GCE node
I20260806 21:09:05.167960 9177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:05.169054 9177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:05.172441 9177 hybrid_clock.cc:648] HybridClock initialized: now 1786050545172365 us; error 119 us; skew 500 ppm
Aug 06 21:09:05 dist-test-slave-69rg krb5kdc[8114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050545, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:05.177098 9177 init.cc:377] Logged in from keytab as kudu/127.27.225.193@KRBTEST.COM (short username kudu)
I20260806 21:09:05.178678 9177 webserver.cc:460] Webserver started at http://127.27.225.193:45571/ using document root <none> and password file <none>
I20260806 21:09:05.179666 9177 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:05.179879 9177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 21:09:05 dist-test-slave-69rg krb5kdc[8114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050545, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for HTTP/127.27.225.212@KRBTEST.COM
I20260806 21:09:05.210273 9177 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.003s
I20260806 21:09:05.218947 9224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:05.220940 9177 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 21:09:05.221108 9177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "634d8826fc8048b8a3732d086431f50e"
format_stamp: "Formatted at 2026-08-06 21:09:03 on dist-test-slave-69rg"
server_key: "c23e5d01d013c50e5ec903e636c59f47"
server_key_iv: "af3ee465e7aff3132ee8816b9b4ab7ac"
server_key_version: "kuduclusterkey@0"
I20260806 21:09:05.221966 9177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:05.263985 9177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:05.268491 9177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:05.269371 9177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:05.270740 9177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:09:05.272410 9177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:09:05.272910 9177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260806 21:09:05.273011 9177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:09:05.273044 9177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:05.288143 9177 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:33225
I20260806 21:09:05.288244 9338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:33225 every 8 connection(s)
I20260806 21:09:05.289937 9177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:09:05.296684 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 9177
I20260806 21:09:05.297356 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.225.212:54835/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:42679
--local_ip_for_outbound_sockets=127.27.225.194
--tserver_master_addrs=127.27.225.254:41611
--webserver_port=33593
--webserver_interface=127.27.225.194
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:43939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.27.225.212:54835/kms with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 21:09:05 dist-test-slave-69rg krb5kdc[8114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050545, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:09:05.322913 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:05.292898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:36837 (local address 127.27.225.254:41611)
0806 21:09:05.294099 (+ 1201us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:05.294246 (+ 147us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:05.296104 (+ 1858us) server_negotiation.cc:407] Connection header received
0806 21:09:05.300061 (+ 3957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:05.300083 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:05.300609 (+ 526us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:05.301023 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:05.303541 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:05.305355 (+ 1814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:05.307344 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:05.308187 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:05.308371 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:05.314597 (+ 6226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:05.314610 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:05.314642 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:05.314712 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:05.317932 (+ 3220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:05.318880 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:05.318888 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:05.318895 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:05.319087 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:05.319582 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:05.319585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:05.319587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:05.320207 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:05.320364 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:05.321219 (+ 855us) server_negotiation.cc:299] Negotiation successful
0806 21:09:05.321637 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1013,"thread_start_us":325,"threads_started":1}
I20260806 21:09:05.325591 9342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:05.295079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41611 (local address 127.27.225.193:36837)
0806 21:09:05.295859 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:05.295933 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:05.299358 (+ 3425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:05.302016 (+ 2658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:05.302026 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:05.302436 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:05.303277 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:05.303302 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:05.305582 (+ 2280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:05.305587 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:05.307145 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:05.307162 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:05.310220 (+ 3058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:05.310232 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:05.310701 (+ 469us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:05.311509 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:05.311534 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:05.313927 (+ 2393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:05.318185 (+ 4258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:05.318194 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:05.318214 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:05.318696 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:05.319229 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:05.319246 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:05.319248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:05.319452 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:05.320447 (+ 995us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:05.320454 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:05.320812 (+ 358us) client_negotiation.cc:769] Sending connection context
0806 21:09:05.323894 (+ 3082us) client_negotiation.cc:240] Negotiation successful
0806 21:09:05.324481 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":204,"threads_started":1}
I20260806 21:09:05.329906 9339 heartbeater.cc:347] Connected to a master server at 127.27.225.254:41611
I20260806 21:09:05.330307 9339 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:05.331481 9339 heartbeater.cc:511] Master 127.27.225.254:41611 requested a full tablet report, sending...
W20260806 21:09:05.336285 9339 heartbeater.cc:644] Failed to heartbeat to 127.27.225.254:41611 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 21:09:05.336468 9339 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:05.336733 9339 heartbeater.cc:511] Master 127.27.225.254:41611 requested a full tablet report, sending...
I20260806 21:09:05.338117 9339 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:05.338339 9339 heartbeater.cc:511] Master 127.27.225.254:41611 requested a full tablet report, sending...
W20260806 21:09:05.339239 9339 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 21:09:05.476827 9343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:05.477298 9343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:05.477355 9343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:05.482719 9343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:05.482810 9343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:05.482885 9343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:09:05.489764 9343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:43939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.27.225.212:54835/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:42679
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=33593
--tserver_master_addrs=127.27.225.254:41611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:05.491170 9343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:05.492333 9343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:05.504611 9352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:05.504654 9350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:05.505220 9343 server_base.cc:1034] running on GCE node
W20260806 21:09:05.505256 9348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:05.505755 9343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:05.506548 9343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:05.507815 9343 hybrid_clock.cc:648] HybridClock initialized: now 1786050545507786 us; error 46 us; skew 500 ppm
Aug 06 21:09:05 dist-test-slave-69rg krb5kdc[8114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050545, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:05.512120 9343 init.cc:377] Logged in from keytab as kudu/127.27.225.194@KRBTEST.COM (short username kudu)
I20260806 21:09:05.513339 9343 webserver.cc:460] Webserver started at http://127.27.225.194:33593/ using document root <none> and password file <none>
I20260806 21:09:05.514631 9343 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:05.514837 9343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 21:09:05 dist-test-slave-69rg krb5kdc[8114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050545, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for HTTP/127.27.225.212@KRBTEST.COM
I20260806 21:09:05.542798 9343 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.004s sys 0.000s
I20260806 21:09:05.547616 9360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:05.550417 9343 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260806 21:09:05.550698 9343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "d00e7abc8a2f43cba94db4596c11529a"
format_stamp: "Formatted at 2026-08-06 21:09:04 on dist-test-slave-69rg"
server_key: "46c6535d986f4cb55244954309a40bca"
server_key_iv: "35e65d812c3824e7842b9344bfd068c4"
server_key_version: "kuduclusterkey@0"
I20260806 21:09:05.552160 9343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:05.587970 9343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:05.594837 9343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:05.595310 9343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:05.596421 9343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:09:05.597678 9343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:09:05.597811 9343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 21:09:05.597929 9343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:09:05.597965 9343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:05.617316 9343 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:42679
I20260806 21:09:05.617334 9475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:42679 every 8 connection(s)
I20260806 21:09:05.619131 9343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:09:05.621315 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 9343
I20260806 21:09:05.621889 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.225.212:54835/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:44015
--local_ip_for_outbound_sockets=127.27.225.195
--tserver_master_addrs=127.27.225.254:41611
--webserver_port=44989
--webserver_interface=127.27.225.195
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:43939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.27.225.212:54835/kms with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 21:09:05 dist-test-slave-69rg krb5kdc[8114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050545, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:09:05.647867 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:05.623673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:33575 (local address 127.27.225.254:41611)
0806 21:09:05.623854 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:05.623859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:05.625317 (+ 1458us) server_negotiation.cc:407] Connection header received
0806 21:09:05.626548 (+ 1231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:05.626552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:05.626620 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:05.626705 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:05.628510 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:05.629593 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:05.632028 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:05.632488 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:05.632600 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:05.642723 (+ 10123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:05.642730 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:05.642733 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:05.642772 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:05.644755 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:05.645685 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:05.645690 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:05.645692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:05.645809 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:05.646325 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:05.646328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:05.646330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:05.646581 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:05.646668 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:05.647478 (+ 810us) server_negotiation.cc:299] Negotiation successful
0806 21:09:05.647660 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 21:09:05.650046 9478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:05.622333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41611 (local address 127.27.225.194:33575)
0806 21:09:05.625094 (+ 2761us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:05.625165 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:05.626288 (+ 1123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:05.627065 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:05.627078 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:05.627502 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:05.628277 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:05.628303 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:05.630041 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:05.630045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:05.631832 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:05.631852 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:05.632892 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:05.632899 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:05.633180 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:05.634031 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:05.634060 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:05.642378 (+ 8318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:05.644973 (+ 2595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:05.644982 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:05.645004 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:05.645435 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:05.645980 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:05.645984 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:05.645986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:05.646200 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:05.646753 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:05.646761 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:05.647133 (+ 372us) client_negotiation.cc:769] Sending connection context
0806 21:09:05.648427 (+ 1294us) client_negotiation.cc:240] Negotiation successful
0806 21:09:05.648852 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2457,"thread_start_us":185,"threads_started":1}
I20260806 21:09:05.653755 9476 heartbeater.cc:347] Connected to a master server at 127.27.225.254:41611
I20260806 21:09:05.654126 9476 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:05.654814 9476 heartbeater.cc:511] Master 127.27.225.254:41611 requested a full tablet report, sending...
W20260806 21:09:05.656837 9476 heartbeater.cc:644] Failed to heartbeat to 127.27.225.254:41611 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 21:09:05.656994 9476 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:05.657202 9476 heartbeater.cc:511] Master 127.27.225.254:41611 requested a full tablet report, sending...
I20260806 21:09:05.658329 9476 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:05.658548 9476 heartbeater.cc:511] Master 127.27.225.254:41611 requested a full tablet report, sending...
W20260806 21:09:05.659828 9476 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 21:09:05.794723 9479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:05.795164 9479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:05.795251 9479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:05.801604 9479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:05.801718 9479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:05.801836 9479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:09:05.809430 9479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:43939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.27.225.212:54835/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:44015
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=44989
--tserver_master_addrs=127.27.225.254:41611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:05.810750 9479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:05.812301 9479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:05.829032 9485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:05.830061 9486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:05.830474 9488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:05.830519 9479 server_base.cc:1034] running on GCE node
I20260806 21:09:05.832103 9479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:05.833503 9479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:05.835170 9479 hybrid_clock.cc:648] HybridClock initialized: now 1786050545835141 us; error 50 us; skew 500 ppm
Aug 06 21:09:05 dist-test-slave-69rg krb5kdc[8114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050545, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:05.841562 9479 init.cc:377] Logged in from keytab as kudu/127.27.225.195@KRBTEST.COM (short username kudu)
I20260806 21:09:05.843333 9479 webserver.cc:460] Webserver started at http://127.27.225.195:44989/ using document root <none> and password file <none>
I20260806 21:09:05.844173 9479 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:05.844298 9479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 21:09:05 dist-test-slave-69rg krb5kdc[8114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050545, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for HTTP/127.27.225.212@KRBTEST.COM
I20260806 21:09:05.883001 9479 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 21:09:05.886763 9497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:05.888952 9479 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 21:09:05.889119 9479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "5cd793a3fcb74f1db65b603d79f71faf"
format_stamp: "Formatted at 2026-08-06 21:09:04 on dist-test-slave-69rg"
server_key: "3fb07b5b9a64454d8386a843bcaef2f7"
server_key_iv: "8b4e9d37365156b88ee6591950cd63fd"
server_key_version: "kuduclusterkey@0"
I20260806 21:09:05.889992 9479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:05.906306 9479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:05.914791 9479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:05.915045 9479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:05.915887 9479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:09:05.917073 9479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:09:05.917150 9479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:05.917405 9479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:09:05.917699 9479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:05.930487 9479 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:44015
I20260806 21:09:05.933109 9479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:09:05.934878 9612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:44015 every 8 connection(s)
I20260806 21:09:05.940855 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 9479
Aug 06 21:09:05 dist-test-slave-69rg krb5kdc[8114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050545, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:09:05.953672 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:05.939409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:38841 (local address 127.27.225.254:41611)
0806 21:09:05.939561 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:05.939567 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:05.939591 (+ 24us) server_negotiation.cc:407] Connection header received
0806 21:09:05.940503 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:05.940507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:05.940569 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:05.940629 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:05.943058 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:05.943733 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:05.945214 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:05.945557 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:05.945653 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:05.949298 (+ 3645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:05.949301 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:05.949304 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:05.949332 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:05.950939 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:05.951693 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:05.951698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:05.951700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:05.951798 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:05.952281 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:05.952285 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:05.952286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:05.952490 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:05.952609 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:05.953327 (+ 718us) server_negotiation.cc:299] Negotiation successful
0806 21:09:05.953482 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 21:09:05.954766 9615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:05.937286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41611 (local address 127.27.225.195:38841)
0806 21:09:05.938486 (+ 1200us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:05.938550 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:05.940052 (+ 1502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:05.941401 (+ 1349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:05.941414 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:05.942071 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:05.942853 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:05.942874 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:05.943879 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:05.943882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:05.945020 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:05.945031 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:05.945965 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:05.945970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:05.946211 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:05.947027 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:05.947053 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:05.949052 (+ 1999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:05.951101 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:05.951109 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:05.951130 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:05.951541 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:05.951923 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:05.951926 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:05.951928 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:05.952141 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:05.952615 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:05.952624 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:05.952967 (+ 343us) client_negotiation.cc:769] Sending connection context
0806 21:09:05.953215 (+ 248us) client_negotiation.cc:240] Negotiation successful
0806 21:09:05.953609 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":934,"thread_start_us":521,"threads_started":1}
I20260806 21:09:05.955960 9613 heartbeater.cc:347] Connected to a master server at 127.27.225.254:41611
I20260806 21:09:05.956220 9613 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:05.956825 9613 heartbeater.cc:511] Master 127.27.225.254:41611 requested a full tablet report, sending...
W20260806 21:09:05.958679 9613 heartbeater.cc:644] Failed to heartbeat to 127.27.225.254:41611 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 21:09:05.958832 9613 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:05.959050 9613 heartbeater.cc:511] Master 127.27.225.254:41611 requested a full tablet report, sending...
I20260806 21:09:05.960067 9613 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:05.960253 9613 heartbeater.cc:511] Master 127.27.225.254:41611 requested a full tablet report, sending...
W20260806 21:09:05.961465 9613 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 21:09:05.963466 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:05.941981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38690 (local address 127.27.225.254:41611)
0806 21:09:05.953938 (+ 11957us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:05.953942 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:05.953956 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:09:05.953994 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:05.953996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:05.954045 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:05.954222 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:05.955222 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:05.956050 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:05.957192 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:05.957537 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:05.957660 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:05.959859 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:05.959863 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:05.959866 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:05.959897 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:05.961495 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:05.962135 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:05.962140 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:05.962143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:05.962236 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:05.962547 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:05.962550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:05.962552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:05.962756 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:05.962978 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:05.963143 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 21:09:05.963269 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11889,"thread_start_us":106,"threads_started":1}
I20260806 21:09:06.339530 9339 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:06.339879 9339 heartbeater.cc:511] Master 127.27.225.254:41611 requested a full tablet report, sending...
I20260806 21:09:06.341248 9137 ts_manager.cc:194] Registered new tserver with Master: 634d8826fc8048b8a3732d086431f50e (127.27.225.193:33225)
I20260806 21:09:06.343084 9137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.193@KRBTEST.COM'} at 127.27.225.193:36837
Aug 06 21:09:06 dist-test-slave-69rg krb5kdc[8114](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786050546, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:06.660118 9476 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:06.660445 9476 heartbeater.cc:511] Master 127.27.225.254:41611 requested a full tablet report, sending...
I20260806 21:09:06.661247 9137 ts_manager.cc:194] Registered new tserver with Master: d00e7abc8a2f43cba94db4596c11529a (127.27.225.194:42679)
I20260806 21:09:06.661948 9137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.194@KRBTEST.COM'} at 127.27.225.194:33575
Aug 06 21:09:06 dist-test-slave-69rg krb5kdc[8114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050546, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for HTTP/127.27.225.212@KRBTEST.COM
I20260806 21:09:06.961726 9613 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:06.962116 9613 heartbeater.cc:511] Master 127.27.225.254:41611 requested a full tablet report, sending...
I20260806 21:09:06.963580 9137 ts_manager.cc:194] Registered new tserver with Master: 5cd793a3fcb74f1db65b603d79f71faf (127.27.225.195:44015)
I20260806 21:09:06.964394 9137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.195@KRBTEST.COM'} at 127.27.225.195:38841
I20260806 21:09:06.973807 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:09:07.002496 9636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:06.987392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38700 (local address 127.27.225.254:41611)
0806 21:09:06.988153 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:06.988158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:06.988172 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:09:06.988218 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:06.988220 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:06.988268 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:06.988860 (+ 592us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:06.989899 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:06.990473 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:06.994390 (+ 3917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:06.994699 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:06.994837 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:06.996697 (+ 1860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:06.996702 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:06.996705 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:06.996743 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:07.000319 (+ 3576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:07.000966 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:07.000970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:07.000972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:07.001079 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:07.001448 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:07.001451 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:07.001453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:07.001700 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:07.001860 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:07.002161 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 21:09:07.002309 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":110,"threads_started":1}
I20260806 21:09:07.007932 9137 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38700:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:09:07.014302 9137 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:09:07.066067 9641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:07.048503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:42679 (local address 127.0.0.1:57942)
0806 21:09:07.054173 (+ 5670us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:07.054369 (+ 196us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:07.054559 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:07.057751 (+ 3192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:07.057758 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:07.057788 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:09:07.058014 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:07.058023 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:07.061083 (+ 3060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:07.061087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:07.062777 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:07.062790 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:07.065410 (+ 2620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:07.065414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:07.065553 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:07.065597 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 21:09:07.065710 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 21:09:07.065879 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5426,"thread_start_us":339,"threads_started":1}
I20260806 21:09:07.067358 9642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:07.050170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57942 (local address 127.27.225.194:42679)
0806 21:09:07.057122 (+ 6952us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:07.057130 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:07.057160 (+ 30us) server_negotiation.cc:407] Connection header received
0806 21:09:07.057254 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:07.057261 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:07.057533 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:07.057739 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:09:07.058209 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:07.060917 (+ 2708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:07.062956 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:07.065036 (+ 2080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:07.065174 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:07.065334 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:07.067055 (+ 1721us) server_negotiation.cc:299] Negotiation successful
0806 21:09:07.067179 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6812,"thread_start_us":128,"threads_started":1}
I20260806 21:09:07.072626 9402 tablet_service.cc:1467] Processing CreateTablet for tablet 4343250ef7f54e16b676e07b291bb103 (DEFAULT_TABLE table=test-table [id=37ab61f708ac4e8a925fad67ca72ecdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:09:07.075843 9402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4343250ef7f54e16b676e07b291bb103. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:09:07.080144 9643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:07.052492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:33225 (local address 127.0.0.1:54752)
0806 21:09:07.062045 (+ 9553us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:07.062062 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:07.062414 (+ 352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:07.064410 (+ 1996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:07.064413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:07.064435 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:09:07.064539 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:07.064546 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:07.066055 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:07.066059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:07.067252 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:07.067265 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:07.079578 (+ 12313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:07.079583 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:07.079716 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:07.079748 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 21:09:07.079842 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 21:09:07.079948 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9450,"thread_start_us":419,"threads_started":1}
I20260806 21:09:07.082561 9641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:07.056503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:44015 (local address 127.0.0.1:46474)
0806 21:09:07.066800 (+ 10297us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:07.066815 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:07.066907 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:07.068142 (+ 1235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:07.068147 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:07.068167 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:09:07.068247 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:07.068253 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:07.070160 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:07.070164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:07.071441 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:07.071456 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:07.082104 (+ 10648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:07.082109 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:07.082218 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:07.082247 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 21:09:07.082324 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 21:09:07.082415 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10226,"thread_start_us":134,"threads_started":1}
I20260806 21:09:07.083271 9646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:07.056567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46474 (local address 127.27.225.195:44015)
0806 21:09:07.063212 (+ 6645us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:07.063222 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:07.067425 (+ 4203us) server_negotiation.cc:407] Connection header received
0806 21:09:07.067525 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:07.067532 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:07.067819 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:07.067989 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:09:07.068393 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:07.069979 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:07.080811 (+ 10832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:07.081736 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:07.081893 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:07.082029 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:07.083000 (+ 971us) server_negotiation.cc:299] Negotiation successful
0806 21:09:07.083123 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6380,"thread_start_us":102,"threads_started":1}
I20260806 21:09:07.083820 9644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:07.053780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54752 (local address 127.27.225.193:33225)
0806 21:09:07.063446 (+ 9666us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:07.063453 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:07.063482 (+ 29us) server_negotiation.cc:407] Connection header received
0806 21:09:07.063571 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:07.063577 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:07.064018 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:07.064224 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:09:07.064687 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:07.065915 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:07.077936 (+ 12021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:07.079117 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:07.079319 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:07.079466 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:07.083556 (+ 4090us) server_negotiation.cc:299] Negotiation successful
0806 21:09:07.083683 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9542,"thread_start_us":85,"threads_started":1}
I20260806 21:09:07.088075 9546 tablet_service.cc:1467] Processing CreateTablet for tablet 4343250ef7f54e16b676e07b291bb103 (DEFAULT_TABLE table=test-table [id=37ab61f708ac4e8a925fad67ca72ecdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:09:07.089531 9546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4343250ef7f54e16b676e07b291bb103. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:09:07.097329 9273 tablet_service.cc:1467] Processing CreateTablet for tablet 4343250ef7f54e16b676e07b291bb103 (DEFAULT_TABLE table=test-table [id=37ab61f708ac4e8a925fad67ca72ecdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:09:07.099045 9273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4343250ef7f54e16b676e07b291bb103. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:09:07.100683 9647 tablet_bootstrap.cc:492] T 4343250ef7f54e16b676e07b291bb103 P d00e7abc8a2f43cba94db4596c11529a: Bootstrap starting.
I20260806 21:09:07.103451 9647 tablet_bootstrap.cc:654] T 4343250ef7f54e16b676e07b291bb103 P d00e7abc8a2f43cba94db4596c11529a: Neither blocks nor log segments found. Creating new log.
I20260806 21:09:07.104643 9647 log.cc:824] T 4343250ef7f54e16b676e07b291bb103 P d00e7abc8a2f43cba94db4596c11529a: Log is configured to *not* fsync() on all Append() calls
I20260806 21:09:07.105662 9648 tablet_bootstrap.cc:492] T 4343250ef7f54e16b676e07b291bb103 P 5cd793a3fcb74f1db65b603d79f71faf: Bootstrap starting.
I20260806 21:09:07.108248 9648 tablet_bootstrap.cc:654] T 4343250ef7f54e16b676e07b291bb103 P 5cd793a3fcb74f1db65b603d79f71faf: Neither blocks nor log segments found. Creating new log.
I20260806 21:09:07.109227 9648 log.cc:824] T 4343250ef7f54e16b676e07b291bb103 P 5cd793a3fcb74f1db65b603d79f71faf: Log is configured to *not* fsync() on all Append() calls
I20260806 21:09:07.112016 9647 tablet_bootstrap.cc:492] T 4343250ef7f54e16b676e07b291bb103 P d00e7abc8a2f43cba94db4596c11529a: No bootstrap required, opened a new log
I20260806 21:09:07.112337 9647 ts_tablet_manager.cc:1397] T 4343250ef7f54e16b676e07b291bb103 P d00e7abc8a2f43cba94db4596c11529a: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.005s
I20260806 21:09:07.117106 9647 raft_consensus.cc:348] T 4343250ef7f54e16b676e07b291bb103 P d00e7abc8a2f43cba94db4596c11529a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d00e7abc8a2f43cba94db4596c11529a" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42679 } } peers { permanent_uuid: "634d8826fc8048b8a3732d086431f50e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 33225 } } peers { permanent_uuid: "5cd793a3fcb74f1db65b603d79f71faf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44015 } }
I20260806 21:09:07.118361 9647 raft_consensus.cc:374] T 4343250ef7f54e16b676e07b291bb103 P d00e7abc8a2f43cba94db4596c11529a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:09:07.118541 9647 raft_consensus.cc:729] T 4343250ef7f54e16b676e07b291bb103 P d00e7abc8a2f43cba94db4596c11529a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d00e7abc8a2f43cba94db4596c11529a, State: Initialized, Role: FOLLOWER
I20260806 21:09:07.119381 9647 consensus_queue.cc:260] T 4343250ef7f54e16b676e07b291bb103 P d00e7abc8a2f43cba94db4596c11529a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d00e7abc8a2f43cba94db4596c11529a" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42679 } } peers { permanent_uuid: "634d8826fc8048b8a3732d086431f50e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 33225 } } peers { permanent_uuid: "5cd793a3fcb74f1db65b603d79f71faf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44015 } }
I20260806 21:09:07.120935 9647 ts_tablet_manager.cc:1428] T 4343250ef7f54e16b676e07b291bb103 P d00e7abc8a2f43cba94db4596c11529a: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260806 21:09:07.121790 9476 heartbeater.cc:503] Master 127.27.225.254:41611 was elected leader, sending a full tablet report...
W20260806 21:09:07.126744 9477 tablet.cc:2369] T 4343250ef7f54e16b676e07b291bb103 P d00e7abc8a2f43cba94db4596c11529a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:09:07.127206 9650 tablet_bootstrap.cc:492] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e: Bootstrap starting.
I20260806 21:09:07.133551 9648 tablet_bootstrap.cc:492] T 4343250ef7f54e16b676e07b291bb103 P 5cd793a3fcb74f1db65b603d79f71faf: No bootstrap required, opened a new log
I20260806 21:09:07.134001 9648 ts_tablet_manager.cc:1397] T 4343250ef7f54e16b676e07b291bb103 P 5cd793a3fcb74f1db65b603d79f71faf: Time spent bootstrapping tablet: real 0.029s user 0.007s sys 0.000s
I20260806 21:09:07.138733 9648 raft_consensus.cc:348] T 4343250ef7f54e16b676e07b291bb103 P 5cd793a3fcb74f1db65b603d79f71faf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d00e7abc8a2f43cba94db4596c11529a" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42679 } } peers { permanent_uuid: "634d8826fc8048b8a3732d086431f50e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 33225 } } peers { permanent_uuid: "5cd793a3fcb74f1db65b603d79f71faf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44015 } }
I20260806 21:09:07.139581 9650 tablet_bootstrap.cc:654] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e: Neither blocks nor log segments found. Creating new log.
I20260806 21:09:07.140718 9650 log.cc:824] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e: Log is configured to *not* fsync() on all Append() calls
I20260806 21:09:07.140914 9648 raft_consensus.cc:374] T 4343250ef7f54e16b676e07b291bb103 P 5cd793a3fcb74f1db65b603d79f71faf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:09:07.141037 9648 raft_consensus.cc:729] T 4343250ef7f54e16b676e07b291bb103 P 5cd793a3fcb74f1db65b603d79f71faf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd793a3fcb74f1db65b603d79f71faf, State: Initialized, Role: FOLLOWER
I20260806 21:09:07.142155 9648 consensus_queue.cc:260] T 4343250ef7f54e16b676e07b291bb103 P 5cd793a3fcb74f1db65b603d79f71faf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d00e7abc8a2f43cba94db4596c11529a" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42679 } } peers { permanent_uuid: "634d8826fc8048b8a3732d086431f50e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 33225 } } peers { permanent_uuid: "5cd793a3fcb74f1db65b603d79f71faf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44015 } }
I20260806 21:09:07.143561 9648 ts_tablet_manager.cc:1428] T 4343250ef7f54e16b676e07b291bb103 P 5cd793a3fcb74f1db65b603d79f71faf: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260806 21:09:07.144012 9613 heartbeater.cc:503] Master 127.27.225.254:41611 was elected leader, sending a full tablet report...
I20260806 21:09:07.150792 9650 tablet_bootstrap.cc:492] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e: No bootstrap required, opened a new log
I20260806 21:09:07.151145 9650 ts_tablet_manager.cc:1397] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.007s
I20260806 21:09:07.155516 9650 raft_consensus.cc:348] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d00e7abc8a2f43cba94db4596c11529a" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42679 } } peers { permanent_uuid: "634d8826fc8048b8a3732d086431f50e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 33225 } } peers { permanent_uuid: "5cd793a3fcb74f1db65b603d79f71faf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44015 } }
I20260806 21:09:07.158249 9650 raft_consensus.cc:374] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:09:07.158404 9650 raft_consensus.cc:729] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 634d8826fc8048b8a3732d086431f50e, State: Initialized, Role: FOLLOWER
I20260806 21:09:07.159132 9650 consensus_queue.cc:260] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d00e7abc8a2f43cba94db4596c11529a" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42679 } } peers { permanent_uuid: "634d8826fc8048b8a3732d086431f50e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 33225 } } peers { permanent_uuid: "5cd793a3fcb74f1db65b603d79f71faf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44015 } }
I20260806 21:09:07.160543 9650 ts_tablet_manager.cc:1428] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260806 21:09:07.160998 9339 heartbeater.cc:503] Master 127.27.225.254:41611 was elected leader, sending a full tablet report...
W20260806 21:09:07.188745 9614 tablet.cc:2369] T 4343250ef7f54e16b676e07b291bb103 P 5cd793a3fcb74f1db65b603d79f71faf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:09:07.224416 9655 raft_consensus.cc:482] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:09:07.226818 9655 raft_consensus.cc:504] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d00e7abc8a2f43cba94db4596c11529a" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42679 } } peers { permanent_uuid: "634d8826fc8048b8a3732d086431f50e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 33225 } } peers { permanent_uuid: "5cd793a3fcb74f1db65b603d79f71faf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44015 } }
I20260806 21:09:07.230742 9655 leader_election.cc:288] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d00e7abc8a2f43cba94db4596c11529a (127.27.225.194:42679), 5cd793a3fcb74f1db65b603d79f71faf (127.27.225.195:44015)
I20260806 21:09:07.241885 9642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:07.229863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:44281 (local address 127.27.225.194:42679)
0806 21:09:07.230028 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:07.230033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:07.231210 (+ 1177us) server_negotiation.cc:407] Connection header received
0806 21:09:07.231440 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:07.231444 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:07.231495 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:07.231551 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:09:07.232220 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:07.233020 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:07.239106 (+ 6086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:07.239751 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:07.239902 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:07.239961 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:07.241596 (+ 1635us) server_negotiation.cc:299] Negotiation successful
0806 21:09:07.241698 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 21:09:07.242642 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:07.229337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:42679 (local address 127.27.225.193:44281)
0806 21:09:07.231106 (+ 1769us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:07.231122 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:07.231301 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:07.231850 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:07.231853 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:07.231880 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:09:07.232045 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:07.232070 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:07.233187 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:07.233191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:07.238889 (+ 5698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:07.238903 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:07.241308 (+ 2405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:07.241312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:07.241428 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:07.241457 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 21:09:07.242366 (+ 909us) client_negotiation.cc:240] Negotiation successful
0806 21:09:07.242469 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1696,"thread_start_us":151,"threads_started":1}
I20260806 21:09:07.244889 9646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:07.230669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:55481 (local address 127.27.225.195:44015)
0806 21:09:07.230811 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:07.230818 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:07.234108 (+ 3290us) server_negotiation.cc:407] Connection header received
0806 21:09:07.234316 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:07.234319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:07.234372 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:07.234419 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:09:07.234819 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:07.235609 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:07.242960 (+ 7351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:07.243636 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:07.243744 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:07.243797 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:07.244658 (+ 861us) server_negotiation.cc:299] Negotiation successful
0806 21:09:07.244762 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 21:09:07.244879 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4343250ef7f54e16b676e07b291bb103" candidate_uuid: "634d8826fc8048b8a3732d086431f50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d00e7abc8a2f43cba94db4596c11529a" is_pre_election: true
I20260806 21:09:07.245254 9423 raft_consensus.cc:2454] T 4343250ef7f54e16b676e07b291bb103 P d00e7abc8a2f43cba94db4596c11529a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 634d8826fc8048b8a3732d086431f50e in term 0.
I20260806 21:09:07.245675 9656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:07.230259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:44015 (local address 127.27.225.193:55481)
0806 21:09:07.234001 (+ 3742us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:07.234018 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:07.234192 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:07.234550 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:07.234553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:07.234568 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:09:07.234698 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:07.234704 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:07.235988 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:07.235992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:07.237634 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:07.237648 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:07.243910 (+ 6262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:07.243914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:07.244021 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:07.244052 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 21:09:07.245414 (+ 1362us) client_negotiation.cc:240] Negotiation successful
0806 21:09:07.245513 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3660,"thread_start_us":96,"threads_started":1}
I20260806 21:09:07.245857 9226 leader_election.cc:302] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 634d8826fc8048b8a3732d086431f50e, d00e7abc8a2f43cba94db4596c11529a; no voters:
I20260806 21:09:07.246407 9655 raft_consensus.cc:2781] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:09:07.246508 9655 raft_consensus.cc:482] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:09:07.246554 9655 raft_consensus.cc:3037] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:09:07.246562 9567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4343250ef7f54e16b676e07b291bb103" candidate_uuid: "634d8826fc8048b8a3732d086431f50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd793a3fcb74f1db65b603d79f71faf" is_pre_election: true
I20260806 21:09:07.246842 9567 raft_consensus.cc:2454] T 4343250ef7f54e16b676e07b291bb103 P 5cd793a3fcb74f1db65b603d79f71faf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 634d8826fc8048b8a3732d086431f50e in term 0.
I20260806 21:09:07.248025 9655 raft_consensus.cc:504] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d00e7abc8a2f43cba94db4596c11529a" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42679 } } peers { permanent_uuid: "634d8826fc8048b8a3732d086431f50e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 33225 } } peers { permanent_uuid: "5cd793a3fcb74f1db65b603d79f71faf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44015 } }
I20260806 21:09:07.249775 9423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4343250ef7f54e16b676e07b291bb103" candidate_uuid: "634d8826fc8048b8a3732d086431f50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d00e7abc8a2f43cba94db4596c11529a"
I20260806 21:09:07.249970 9423 raft_consensus.cc:3037] T 4343250ef7f54e16b676e07b291bb103 P d00e7abc8a2f43cba94db4596c11529a [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:09:07.251595 9423 raft_consensus.cc:2454] T 4343250ef7f54e16b676e07b291bb103 P d00e7abc8a2f43cba94db4596c11529a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 634d8826fc8048b8a3732d086431f50e in term 1.
I20260806 21:09:07.252116 9226 leader_election.cc:302] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 634d8826fc8048b8a3732d086431f50e, d00e7abc8a2f43cba94db4596c11529a; no voters:
I20260806 21:09:07.253275 9567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4343250ef7f54e16b676e07b291bb103" candidate_uuid: "634d8826fc8048b8a3732d086431f50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd793a3fcb74f1db65b603d79f71faf"
I20260806 21:09:07.253448 9567 raft_consensus.cc:3037] T 4343250ef7f54e16b676e07b291bb103 P 5cd793a3fcb74f1db65b603d79f71faf [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:09:07.254976 9567 raft_consensus.cc:2454] T 4343250ef7f54e16b676e07b291bb103 P 5cd793a3fcb74f1db65b603d79f71faf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 634d8826fc8048b8a3732d086431f50e in term 1.
I20260806 21:09:07.255759 9655 leader_election.cc:288] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e [CANDIDATE]: Term 1 election: Requested vote from peers d00e7abc8a2f43cba94db4596c11529a (127.27.225.194:42679), 5cd793a3fcb74f1db65b603d79f71faf (127.27.225.195:44015)
I20260806 21:09:07.256265 9655 raft_consensus.cc:2781] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:09:07.256471 9655 raft_consensus.cc:686] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e [term 1 LEADER]: Becoming Leader. State: Replica: 634d8826fc8048b8a3732d086431f50e, State: Running, Role: LEADER
I20260806 21:09:07.257041 9655 consensus_queue.cc:237] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d00e7abc8a2f43cba94db4596c11529a" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42679 } } peers { permanent_uuid: "634d8826fc8048b8a3732d086431f50e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 33225 } } peers { permanent_uuid: "5cd793a3fcb74f1db65b603d79f71faf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44015 } }
I20260806 21:09:07.261265 9137 catalog_manager.cc:5495] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e reported cstate change: term changed from 0 to 1, leader changed from <none> to 634d8826fc8048b8a3732d086431f50e (127.27.225.193). New cstate: current_term: 1 leader_uuid: "634d8826fc8048b8a3732d086431f50e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d00e7abc8a2f43cba94db4596c11529a" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "634d8826fc8048b8a3732d086431f50e" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 33225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd793a3fcb74f1db65b603d79f71faf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44015 } health_report { overall_health: UNKNOWN } } }
W20260806 21:09:07.293916 9340 tablet.cc:2369] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:09:07.302016 9644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:07.296378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54758 (local address 127.27.225.193:33225)
0806 21:09:07.296507 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:07.296512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:07.296526 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:09:07.296614 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:07.296617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:07.296698 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:07.296813 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:09:07.297233 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:07.297999 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:07.299298 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:07.299612 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:07.299711 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:07.300162 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:09:07.301081 (+ 919us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:09:07.301215 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:07.301747 (+ 532us) server_negotiation.cc:299] Negotiation successful
0806 21:09:07.301842 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 21:09:07.312353 9567 raft_consensus.cc:1261] T 4343250ef7f54e16b676e07b291bb103 P 5cd793a3fcb74f1db65b603d79f71faf [term 1 FOLLOWER]: Refusing update from remote peer 634d8826fc8048b8a3732d086431f50e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:09:07.313586 9658 consensus_queue.cc:1035] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd793a3fcb74f1db65b603d79f71faf" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 44015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:09:07.323882 9423 raft_consensus.cc:1261] T 4343250ef7f54e16b676e07b291bb103 P d00e7abc8a2f43cba94db4596c11529a [term 1 FOLLOWER]: Refusing update from remote peer 634d8826fc8048b8a3732d086431f50e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:09:07.324657 9658 consensus_queue.cc:1035] T 4343250ef7f54e16b676e07b291bb103 P 634d8826fc8048b8a3732d086431f50e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d00e7abc8a2f43cba94db4596c11529a" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Aug 06 21:09:07 dist-test-slave-69rg krb5kdc[8114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050546, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for HTTP/127.27.225.212@KRBTEST.COM
I20260806 21:09:07.346333 9662 mvcc.cc:205] Tried to move back new op lower bound from 7315663041780736000 to 7315663041569640448. Current Snapshot: MvccSnapshot[applied={T|T < 7315663041780736000}]
I20260806 21:09:07.357522 9665 mvcc.cc:205] Tried to move back new op lower bound from 7315663041780736000 to 7315663041569640448. Current Snapshot: MvccSnapshot[applied={T|T < 7315663041780736000}]
I20260806 21:09:07.360147 9663 mvcc.cc:205] Tried to move back new op lower bound from 7315663041780736000 to 7315663041569640448. Current Snapshot: MvccSnapshot[applied={T|T < 7315663041780736000}]
I20260806 21:09:07.372757 9137 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:09:07.372990 9137 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:09:07.376435 9137 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44d7afc671e54c36935efafee3b9ca62: Sending DeleteTablet for 3 replicas of tablet 4343250ef7f54e16b676e07b291bb103
I20260806 21:09:07.378647 9402 tablet_service.cc:1514] Processing DeleteTablet for tablet 4343250ef7f54e16b676e07b291bb103 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:09:07 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:57942
I20260806 21:09:07.379738 9273 tablet_service.cc:1514] Processing DeleteTablet for tablet 4343250ef7f54e16b676e07b291bb103 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:09:07 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:54752
I20260806 21:09:07.380858 9546 tablet_service.cc:1514] Processing DeleteTablet for tablet 4343250ef7f54e16b676e07b291bb103 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:09:07 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:46474
I20260806 21:09:07.382009 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 9177
I20260806 21:09:07.386121 9673 tablet_replica.cc:330] T 4343250ef7f54e16b676e07b291bb103 P d00e7abc8a2f43cba94db4596c11529a: stopping tablet replica
I20260806 21:09:07.386613 9673 raft_consensus.cc:2229] T 4343250ef7f54e16b676e07b291bb103 P d00e7abc8a2f43cba94db4596c11529a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:09:07.387040 9673 raft_consensus.cc:2258] T 4343250ef7f54e16b676e07b291bb103 P d00e7abc8a2f43cba94db4596c11529a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:09:07.389058 9673 ts_tablet_manager.cc:1905] T 4343250ef7f54e16b676e07b291bb103 P d00e7abc8a2f43cba94db4596c11529a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:09:07.390692 9676 tablet_replica.cc:330] T 4343250ef7f54e16b676e07b291bb103 P 5cd793a3fcb74f1db65b603d79f71faf: stopping tablet replica
I20260806 21:09:07.391058 9676 raft_consensus.cc:2229] T 4343250ef7f54e16b676e07b291bb103 P 5cd793a3fcb74f1db65b603d79f71faf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:09:07.391499 9676 raft_consensus.cc:2258] T 4343250ef7f54e16b676e07b291bb103 P 5cd793a3fcb74f1db65b603d79f71faf [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 21:09:07.395051 9122 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.225.193:33225 (error 108)
I20260806 21:09:07.396694 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 9343
I20260806 21:09:07.396720 9641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:07.396385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:33225 (local address 127.0.0.1:54770)
0806 21:09:07.396528 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:07.396618 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.193:33225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 21:09:07.397667 9676 ts_tablet_manager.cc:1905] T 4343250ef7f54e16b676e07b291bb103 P 5cd793a3fcb74f1db65b603d79f71faf: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 21:09:07.400858 9122 catalog_manager.cc:4584] TS 634d8826fc8048b8a3732d086431f50e (127.27.225.193:33225): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4343250ef7f54e16b676e07b291bb103: Network error: Client connection negotiation failed: client connection to 127.27.225.193:33225: connect: Connection refused (error 111)
I20260806 21:09:07.401136 9676 ts_tablet_manager.cc:1918] T 4343250ef7f54e16b676e07b291bb103 P 5cd793a3fcb74f1db65b603d79f71faf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:09:07.401271 9676 log.cc:1196] T 4343250ef7f54e16b676e07b291bb103 P 5cd793a3fcb74f1db65b603d79f71faf: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/4343250ef7f54e16b676e07b291bb103
I20260806 21:09:07.401650 9676 ts_tablet_manager.cc:1939] T 4343250ef7f54e16b676e07b291bb103 P 5cd793a3fcb74f1db65b603d79f71faf: Deleting consensus metadata
I20260806 21:09:07.403291 9123 catalog_manager.cc:4841] TS 5cd793a3fcb74f1db65b603d79f71faf (127.27.225.195:44015): tablet 4343250ef7f54e16b676e07b291bb103 (table test-table [id=37ab61f708ac4e8a925fad67ca72ecdd]) successfully deleted
W20260806 21:09:07.410427 9641 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:09:07.409880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:42679 (local address 127.0.0.1:57950)
0806 21:09:07.410022 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:07.410039 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:07.410154 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:07.410275 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.194:42679: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 21:09:07.410559 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 9479
I20260806 21:09:07.423487 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 9103
I20260806 21:09:07.432502 28551 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 21:09:07.779628 28551 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 21:09:07.779765 28551 mini_ranger.cc:66] Stopping Ranger...
I20260806 21:09:08.127607 28551 mini_ranger.cc:68] Stopped Ranger
2026-08-06 21:09:08.128 UTC [8145] LOG: received smart shutdown request
2026-08-06 21:09:08.134 UTC [8145] LOG: background worker "logical replication launcher" (PID 8154) exited with exit code 1
2026-08-06 21:09:08.134 UTC [8149] LOG: shutting down
2026-08-06 21:09:08.478 UTC [8145] LOG: database system is shut down
2026-08-06T21:09:08Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51947 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:09:08 dist-test-slave-69rg krb5kdc[9687](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:09:08 dist-test-slave-69rg krb5kdc[9687](info): set up 2 sockets
Aug 06 21:09:08 dist-test-slave-69rg krb5kdc[9687](info): commencing operation
krb5kdc: starting...
W20260806 21:09:09.792006 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:09:09 dist-test-slave-69rg krb5kdc[9687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:09:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:09:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:09:09.943085 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:37303
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:44655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:37303,127.27.225.253:46611,127.27.225.252:37579
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:09:10.075443 9702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:10.075766 9702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:10.075812 9702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:10.080327 9702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:09:10.080420 9702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:10.080443 9702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:10.080461 9702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:09:10.080478 9702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:09:10.089927 9702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:44655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:37303,127.27.225.253:46611,127.27.225.252:37579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:37303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:10.091260 9702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:10.092427 9702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:10.102638 9707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:10.102658 9708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:10.102660 9710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:10.103228 9702 server_base.cc:1034] running on GCE node
I20260806 21:09:10.103940 9702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:10.105278 9702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:10.106508 9702 hybrid_clock.cc:648] HybridClock initialized: now 1786050550106462 us; error 73 us; skew 500 ppm
Aug 06 21:09:10 dist-test-slave-69rg krb5kdc[9687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050550, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:10.109717 9702 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:09:10.110723 9702 webserver.cc:460] Webserver started at http://127.27.225.254:43799/ using document root <none> and password file <none>
I20260806 21:09:10.111268 9702 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:10.111332 9702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:10.111505 9702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:09:10.113591 9702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "b31df38ec7724a53a55203ed9e131c84"
format_stamp: "Formatted at 2026-08-06 21:09:10 on dist-test-slave-69rg"
server_key: "775c1e89dcd06cb8c6e554fee42a396d"
server_key_iv: "5d5c371967854a584ce9c94dcbd861a3"
server_key_version: "encryptionkey@0"
I20260806 21:09:10.114212 9702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "b31df38ec7724a53a55203ed9e131c84"
format_stamp: "Formatted at 2026-08-06 21:09:10 on dist-test-slave-69rg"
server_key: "775c1e89dcd06cb8c6e554fee42a396d"
server_key_iv: "5d5c371967854a584ce9c94dcbd861a3"
server_key_version: "encryptionkey@0"
I20260806 21:09:10.118157 9702 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:09:10.120908 9717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:10.122457 9702 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:09:10.122601 9702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "b31df38ec7724a53a55203ed9e131c84"
format_stamp: "Formatted at 2026-08-06 21:09:10 on dist-test-slave-69rg"
server_key: "775c1e89dcd06cb8c6e554fee42a396d"
server_key_iv: "5d5c371967854a584ce9c94dcbd861a3"
server_key_version: "encryptionkey@0"
I20260806 21:09:10.122711 9702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:10.145807 9702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:10.149449 9702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:10.149632 9702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:10.157689 9702 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:37303
I20260806 21:09:10.158057 9770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:37303 every 8 connection(s)
I20260806 21:09:10.158849 9702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:09:10.161808 9771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:09:10.162528 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 9702
I20260806 21:09:10.162734 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:09:10.163054 28551 external_mini_cluster.cc:1428] Setting key 5d7634a3f6fa4692eccf7ed4ce001347
I20260806 21:09:10.166901 9771 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37303 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.27.225.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.253@KRBTEST.COM" created.
Aug 06 21:09:10 dist-test-slave-69rg krb5kdc[9687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050550, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:09:10.185488 9776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.170121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57144 (local address 127.27.225.254:37303)
0806 21:09:10.170414 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:10.170424 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:10.170461 (+ 37us) server_negotiation.cc:407] Connection header received
0806 21:09:10.171691 (+ 1230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:10.171696 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:10.171900 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:10.172114 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:10.173203 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.174793 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.176014 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.176622 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.176798 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:10.179728 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:10.179735 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:10.179747 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:10.179777 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:10.181696 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.182448 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.182453 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.182454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.182571 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.183093 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.183096 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.183097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.183583 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:10.183707 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:10.184293 (+ 586us) server_negotiation.cc:299] Negotiation successful
0806 21:09:10.184533 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":201,"thread_start_us":114,"threads_started":1}
I20260806 21:09:10.185492 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.169294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:37303 (local address 127.0.0.1:57144)
0806 21:09:10.169939 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.170004 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:10.171325 (+ 1321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:10.172134 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:10.172141 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:10.172664 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:10.172970 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:10.172985 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.175000 (+ 2015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.175003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:10.175867 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:10.175879 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.176826 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.176830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:10.176980 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:10.177740 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:10.177758 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:10.179538 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:10.181865 (+ 2327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:10.181878 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:10.181905 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:10.182297 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:10.182760 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:10.182767 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:10.182772 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:10.182952 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:10.183732 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:10.183741 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:10.183952 (+ 211us) client_negotiation.cc:769] Sending connection context
0806 21:09:10.184173 (+ 221us) client_negotiation.cc:240] Negotiation successful
0806 21:09:10.184638 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":60,"thread_start_us":130,"threads_started":1}
I20260806 21:09:10.188877 9771 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:09:10.190196 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.189572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:46611 (local address 127.0.0.1:50826)
0806 21:09:10.189950 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.190063 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.253:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280}
W20260806 21:09:10.190476 9719 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.225.253:46611: connect: Connection refused (error 111)
I20260806 21:09:10.191550 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.191321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:46611 (local address 127.0.0.1:50832)
0806 21:09:10.191440 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.191489 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.253:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 21:09:10.191807 9771 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.253:46611: Network error: Client connection negotiation failed: client connection to 127.27.225.253:46611: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.253.keytab.
Entry for principal kudu/127.27.225.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.253.keytab.
I20260806 21:09:10.204854 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.253:46611
--webserver_interface=127.27.225.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.253.keytab
--principal=kudu/127.27.225.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:44655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:37303,127.27.225.253:46611,127.27.225.252:37579
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:09:10.241122 9771 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } attempt: 1
I20260806 21:09:10.242199 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.241869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:46611 (local address 127.0.0.1:50834)
0806 21:09:10.242036 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.242111 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.253:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 21:09:10.243166 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.242922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:46611 (local address 127.0.0.1:50836)
0806 21:09:10.243040 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.243095 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.253:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 21:09:10.243302 9771 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.253:46611: Network error: Client connection negotiation failed: client connection to 127.27.225.253:46611: connect: Connection refused (error 111)
I20260806 21:09:10.311494 9771 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } attempt: 2
I20260806 21:09:10.312608 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.312212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:46611 (local address 127.0.0.1:50844)
0806 21:09:10.312462 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.312527 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.253:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 21:09:10.313454 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.313210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:46611 (local address 127.0.0.1:50858)
0806 21:09:10.313339 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.313396 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.253:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 21:09:10.313601 9771 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.253:46611: Network error: Client connection negotiation failed: client connection to 127.27.225.253:46611: connect: Connection refused (error 111)
W20260806 21:09:10.336370 9779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:10.336736 9779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:10.336820 9779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:10.341990 9779 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:09:10.342136 9779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:10.342175 9779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:10.342219 9779 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:09:10.342258 9779 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:09:10.351110 9779 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:44655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:37303,127.27.225.253:46611,127.27.225.252:37579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.253.keytab
--principal=kudu/127.27.225.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.253:46611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.225.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:10.352442 9779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:10.353546 9779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:10.363268 9787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:10.363397 9779 server_base.cc:1034] running on GCE node
W20260806 21:09:10.363338 9784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:10.363268 9785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:10.364693 9779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:10.366509 9779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:10.367771 9779 hybrid_clock.cc:648] HybridClock initialized: now 1786050550367734 us; error 59 us; skew 500 ppm
Aug 06 21:09:10 dist-test-slave-69rg krb5kdc[9687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050550, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:10.371804 9779 init.cc:377] Logged in from keytab as kudu/127.27.225.253@KRBTEST.COM (short username kudu)
I20260806 21:09:10.372881 9779 webserver.cc:460] Webserver started at http://127.27.225.253:34511/ using document root <none> and password file <none>
I20260806 21:09:10.373499 9779 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:10.373571 9779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:10.373802 9779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:09:10.376107 9779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-1/data/instance:
uuid: "9ade41d137484cdfa993675c915c6ca6"
format_stamp: "Formatted at 2026-08-06 21:09:10 on dist-test-slave-69rg"
server_key: "c02c22ffb2c54b8bd672a9164e70e784"
server_key_iv: "874bb7972cd30aa085449ef60b6e3b00"
server_key_version: "encryptionkey@0"
I20260806 21:09:10.376761 9779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-1/wal/instance:
uuid: "9ade41d137484cdfa993675c915c6ca6"
format_stamp: "Formatted at 2026-08-06 21:09:10 on dist-test-slave-69rg"
server_key: "c02c22ffb2c54b8bd672a9164e70e784"
server_key_iv: "874bb7972cd30aa085449ef60b6e3b00"
server_key_version: "encryptionkey@0"
I20260806 21:09:10.381731 9779 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260806 21:09:10.385460 9794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:10.387778 9779 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260806 21:09:10.388062 9779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-1/wal
uuid: "9ade41d137484cdfa993675c915c6ca6"
format_stamp: "Formatted at 2026-08-06 21:09:10 on dist-test-slave-69rg"
server_key: "c02c22ffb2c54b8bd672a9164e70e784"
server_key_iv: "874bb7972cd30aa085449ef60b6e3b00"
server_key_version: "encryptionkey@0"
I20260806 21:09:10.388309 9779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:10.404801 9771 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } attempt: 3
I20260806 21:09:10.405941 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.405462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:46611 (local address 127.0.0.1:50872)
0806 21:09:10.405706 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.405790 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.253:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 21:09:10.406754 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.406514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:46611 (local address 127.0.0.1:50880)
0806 21:09:10.406634 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.406696 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.253:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 21:09:10.406932 9771 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.253:46611: Network error: Client connection negotiation failed: client connection to 127.27.225.253:46611: connect: Connection refused (error 111)
I20260806 21:09:10.409132 9779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:10.413901 9779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:10.414140 9779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:10.423811 9779 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.253:46611
I20260806 21:09:10.424130 9848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.253:46611 every 8 connection(s)
I20260806 21:09:10.425170 9779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-1/data/info.pb
I20260806 21:09:10.429006 9849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:09:10.434190 9849 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37303 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:09:10.435205 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 9779
I20260806 21:09:10.435313 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-1/wal/instance
I20260806 21:09:10.435705 28551 external_mini_cluster.cc:1428] Setting key ea0608d598ef61a1fc58833c645acdae
WARNING: no policy specified for kudu/127.27.225.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.252@KRBTEST.COM" created.
Aug 06 21:09:10 dist-test-slave-69rg krb5kdc[9687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050550, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.253@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:09:10.457926 9776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.436886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57148 (local address 127.27.225.254:37303)
0806 21:09:10.437237 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:10.437242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:10.438528 (+ 1286us) server_negotiation.cc:407] Connection header received
0806 21:09:10.439581 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:10.439584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:10.439764 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:10.439906 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:10.441437 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.442356 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.443936 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.444216 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.444354 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:10.452233 (+ 7879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:10.452237 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:10.452239 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:10.452272 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:10.454794 (+ 2522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.455540 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.455546 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.455551 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.455666 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.456167 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.456174 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.456179 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.456488 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:10.456624 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:10.457387 (+ 763us) server_negotiation.cc:299] Negotiation successful
0806 21:09:10.457565 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260806 21:09:10.458822 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.437475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:37303 (local address 127.0.0.1:57148)
0806 21:09:10.438055 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.438372 (+ 317us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:10.439353 (+ 981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:10.440115 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:10.440123 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:10.440633 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:10.441212 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:10.441229 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.442559 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.442565 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:10.443534 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:10.443545 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.444414 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.444418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:10.444619 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:10.445406 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:10.445433 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:10.451962 (+ 6529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:10.455029 (+ 3067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:10.455036 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:10.455053 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:10.455404 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:10.455873 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:10.455876 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:10.455878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:10.456046 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:10.456741 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:10.456751 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:10.457076 (+ 325us) client_negotiation.cc:769] Sending connection context
0806 21:09:10.457261 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 21:09:10.457562 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":179,"threads_started":1}
I20260806 21:09:10.460680 9849 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 21:09:10 dist-test-slave-69rg krb5kdc[9687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050550, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.253@KRBTEST.COM for kudu/127.27.225.253@KRBTEST.COM
I20260806 21:09:10.475212 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.461431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:46611 (local address 127.0.0.1:50894)
0806 21:09:10.461601 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.461616 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:10.461702 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:10.462603 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:10.462606 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:10.463038 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:10.463171 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:10.463179 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.464594 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.464596 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:10.465413 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:10.465423 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.466626 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.466631 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:10.466752 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:10.467297 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:10.467310 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:10.468856 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:10.472416 (+ 3560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:10.472421 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:10.472424 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:10.472856 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:10.473306 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:10.473309 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:10.473310 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:10.473651 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:10.474496 (+ 845us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:10.474499 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:10.474594 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 21:09:10.474902 (+ 308us) client_negotiation.cc:240] Negotiation successful
0806 21:09:10.475063 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 21:09:10.475479 9854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.461585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50894 (local address 127.27.225.253:46611)
0806 21:09:10.461952 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:10.462110 (+ 158us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:10.462134 (+ 24us) server_negotiation.cc:407] Connection header received
0806 21:09:10.462176 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:10.462181 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:10.462358 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:10.462485 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:10.463401 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.464430 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.465623 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.466297 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.466396 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:10.469094 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:10.469108 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:10.469135 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:10.469158 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:10.471934 (+ 2776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.473030 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.473036 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.473038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.473142 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.473793 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.473796 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.473797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.474294 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:10.474413 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:10.475039 (+ 626us) server_negotiation.cc:299] Negotiation successful
0806 21:09:10.475182 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":170,"threads_started":1}
I20260806 21:09:10.477428 9849 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:09:10.478577 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.478268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:37579 (local address 127.0.0.1:53532)
0806 21:09:10.478407 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.478505 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 21:09:10.478888 9795 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.225.252:37579: connect: Connection refused (error 111)
I20260806 21:09:10.480357 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.479950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:37579 (local address 127.0.0.1:53548)
0806 21:09:10.480114 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.480237 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260806 21:09:10.480736 9849 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.252:37579: Network error: Client connection negotiation failed: client connection to 127.27.225.252:37579: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.252.keytab.
Entry for principal kudu/127.27.225.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.252.keytab.
I20260806 21:09:10.484736 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.252:37579
--webserver_interface=127.27.225.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.252.keytab
--principal=kudu/127.27.225.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:44655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:37303,127.27.225.253:46611,127.27.225.252:37579
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:09:10.530460 9849 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } attempt: 1
I20260806 21:09:10.531658 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.531267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:37579 (local address 127.0.0.1:53554)
0806 21:09:10.531507 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.531572 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260806 21:09:10.532835 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.532558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:37579 (local address 127.0.0.1:53570)
0806 21:09:10.532702 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.532765 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260806 21:09:10.532989 9849 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.252:37579: Network error: Client connection negotiation failed: client connection to 127.27.225.252:37579: connect: Connection refused (error 111)
I20260806 21:09:10.550190 9771 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } attempt: 4
Aug 06 21:09:10 dist-test-slave-69rg krb5kdc[9687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050550, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for kudu/127.27.225.253@KRBTEST.COM
I20260806 21:09:10.561250 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.550894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:46611 (local address 127.0.0.1:50902)
0806 21:09:10.551099 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.551119 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:10.551233 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:10.551743 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:10.551750 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:10.551948 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:10.552070 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:10.552077 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.553174 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.553177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:10.554487 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:10.554502 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.555181 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.555184 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:10.555290 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:10.555876 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:10.555891 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:10.557465 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:10.559666 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:10.559669 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:10.559671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:10.559902 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:10.560214 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:10.560216 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:10.560217 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:10.560303 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:10.560759 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:10.560761 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:10.560861 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 21:09:10.560946 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 21:09:10.561113 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260806 21:09:10.561316 9854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.551009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50902 (local address 127.27.225.253:46611)
0806 21:09:10.551160 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:10.551165 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:10.551185 (+ 20us) server_negotiation.cc:407] Connection header received
0806 21:09:10.551359 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:10.551363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:10.551425 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:10.551553 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:10.552217 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.553027 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.554635 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.555007 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.555123 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:10.557644 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:10.557648 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:10.557651 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:10.557685 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:10.559518 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.560009 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.560012 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.560014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.560110 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.560398 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.560401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.560402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.560639 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:10.560748 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:10.561026 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 21:09:10.561161 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 21:09:10.562503 9771 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:09:10.563330 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.563049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:37579 (local address 127.0.0.1:53582)
0806 21:09:10.563191 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.563250 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 21:09:10.564020 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.563811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:37579 (local address 127.0.0.1:53592)
0806 21:09:10.563915 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.563962 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 21:09:10.564245 9771 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.252:37579: Network error: Client connection negotiation failed: client connection to 127.27.225.252:37579: connect: Connection refused (error 111)
I20260806 21:09:10.601212 9849 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } attempt: 2
I20260806 21:09:10.602319 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.602006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:37579 (local address 127.0.0.1:53606)
0806 21:09:10.602163 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.602236 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 21:09:10.603106 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.602887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:37579 (local address 127.0.0.1:53620)
0806 21:09:10.602996 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.603043 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 21:09:10.603258 9849 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.252:37579: Network error: Client connection negotiation failed: client connection to 127.27.225.252:37579: connect: Connection refused (error 111)
I20260806 21:09:10.623451 9771 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } attempt: 1
I20260806 21:09:10.624775 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.624208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:37579 (local address 127.0.0.1:53622)
0806 21:09:10.624443 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.624517 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260806 21:09:10.625701 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.625463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:37579 (local address 127.0.0.1:53626)
0806 21:09:10.625581 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.625640 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 21:09:10.625903 9771 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.252:37579: Network error: Client connection negotiation failed: client connection to 127.27.225.252:37579: connect: Connection refused (error 111)
W20260806 21:09:10.637367 9856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:10.637914 9856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:10.638015 9856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:10.644279 9856 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:09:10.644367 9856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:10.644392 9856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:10.644411 9856 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:09:10.644429 9856 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:09:10.650895 9856 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:44655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:37303,127.27.225.253:46611,127.27.225.252:37579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.252.keytab
--principal=kudu/127.27.225.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.252:37579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.225.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:10.652383 9856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:10.653782 9856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:10.664372 9861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:10.664356 9864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:10.664356 9862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:10.665105 9856 server_base.cc:1034] running on GCE node
I20260806 21:09:10.665776 9856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:10.667155 9856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:10.668390 9856 hybrid_clock.cc:648] HybridClock initialized: now 1786050550668353 us; error 53 us; skew 500 ppm
Aug 06 21:09:10 dist-test-slave-69rg krb5kdc[9687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050550, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:10.675295 9856 init.cc:377] Logged in from keytab as kudu/127.27.225.252@KRBTEST.COM (short username kudu)
I20260806 21:09:10.676589 9856 webserver.cc:460] Webserver started at http://127.27.225.252:37231/ using document root <none> and password file <none>
I20260806 21:09:10.677248 9856 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:10.677328 9856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:10.677517 9856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:09:10.680621 9856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-2/data/instance:
uuid: "30c2a02670404bb1877f474ba1036631"
format_stamp: "Formatted at 2026-08-06 21:09:10 on dist-test-slave-69rg"
server_key: "767473ce8c0562bb940d6bb2bdc3c3ff"
server_key_iv: "ac93667b65e09190678512b3a18b8f94"
server_key_version: "encryptionkey@0"
I20260806 21:09:10.681358 9856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-2/wal/instance:
uuid: "30c2a02670404bb1877f474ba1036631"
format_stamp: "Formatted at 2026-08-06 21:09:10 on dist-test-slave-69rg"
server_key: "767473ce8c0562bb940d6bb2bdc3c3ff"
server_key_iv: "ac93667b65e09190678512b3a18b8f94"
server_key_version: "encryptionkey@0"
I20260806 21:09:10.687052 9856 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 21:09:10.691514 9871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:10.693111 9771 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } attempt: 2
I20260806 21:09:10.693523 9856 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260806 21:09:10.693677 9856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-2/wal
uuid: "30c2a02670404bb1877f474ba1036631"
format_stamp: "Formatted at 2026-08-06 21:09:10 on dist-test-slave-69rg"
server_key: "767473ce8c0562bb940d6bb2bdc3c3ff"
server_key_iv: "ac93667b65e09190678512b3a18b8f94"
server_key_version: "encryptionkey@0"
I20260806 21:09:10.693861 9856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:10.694159 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.693764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:37579 (local address 127.0.0.1:53638)
0806 21:09:10.693963 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.694062 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 21:09:10.694487 9849 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } attempt: 3
I20260806 21:09:10.695286 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.694930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:37579 (local address 127.0.0.1:53650)
0806 21:09:10.695099 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.695162 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 21:09:10.695412 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.694812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:37579 (local address 127.0.0.1:53642)
0806 21:09:10.694925 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.694987 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 21:09:10.696025 9771 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.252:37579: Network error: Client connection negotiation failed: client connection to 127.27.225.252:37579: connect: Connection refused (error 111)
I20260806 21:09:10.696437 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.696155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:37579 (local address 127.0.0.1:53666)
0806 21:09:10.696309 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.696363 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260806 21:09:10.696650 9849 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.252:37579: Network error: Client connection negotiation failed: client connection to 127.27.225.252:37579: connect: Connection refused (error 111)
I20260806 21:09:10.710326 9856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:10.715335 9856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:10.715548 9856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:10.726686 9856 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.252:37579
I20260806 21:09:10.727246 9924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.252:37579 every 8 connection(s)
I20260806 21:09:10.728066 9856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-2/data/info.pb
I20260806 21:09:10.731432 9925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:09:10.735805 9925 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37303 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:09:10.736630 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 9856
I20260806 21:09:10.736783 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050378084738-28551-0/minicluster-data/master-2/wal/instance
I20260806 21:09:10.737074 28551 external_mini_cluster.cc:1428] Setting key 5c5e59e4a62f4891be27419897e9e9d5
Aug 06 21:09:10 dist-test-slave-69rg krb5kdc[9687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050550, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.252@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
Aug 06 21:09:10 dist-test-slave-69rg krb5kdc[9687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.252@KRBTEST.COM
I20260806 21:09:10.759449 9776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.738514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57150 (local address 127.27.225.254:37303)
0806 21:09:10.738844 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:10.738850 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:10.739838 (+ 988us) server_negotiation.cc:407] Connection header received
0806 21:09:10.741398 (+ 1560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:10.741402 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:10.741461 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:10.741565 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:10.743968 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.744762 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.746528 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.746927 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.747050 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:10.751770 (+ 4720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:10.751780 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:10.751786 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:10.751833 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:10.755053 (+ 3220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.756226 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.756229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.756230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.756708 (+ 478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.757091 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.757095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.757096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.757313 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:10.757873 (+ 560us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:10.759125 (+ 1252us) server_negotiation.cc:299] Negotiation successful
0806 21:09:10.759269 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226}
I20260806 21:09:10.760185 9928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.738893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:37303 (local address 127.0.0.1:57150)
0806 21:09:10.739617 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.739680 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:10.740592 (+ 912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:10.741800 (+ 1208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:10.741807 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:10.743103 (+ 1296us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:10.743695 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:10.743716 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.744926 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.744929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:10.746275 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:10.746287 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.747120 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.747123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:10.747290 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:10.748312 (+ 1022us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:10.748336 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:10.751445 (+ 3109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:10.755754 (+ 4309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:10.755766 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:10.755778 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:10.756051 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:10.756848 (+ 797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:10.756853 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:10.756855 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:10.756971 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:10.757618 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:10.757890 (+ 272us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:10.758472 (+ 582us) client_negotiation.cc:769] Sending connection context
0806 21:09:10.758982 (+ 510us) client_negotiation.cc:240] Negotiation successful
0806 21:09:10.759525 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"spinlock_wait_cycles":768,"thread_start_us":175,"threads_started":1}
I20260806 21:09:10.760190 9929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.738821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53674 (local address 127.27.225.252:37579)
0806 21:09:10.739569 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:10.739583 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:10.739838 (+ 255us) server_negotiation.cc:407] Connection header received
0806 21:09:10.740696 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:10.740776 (+ 80us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:10.741377 (+ 601us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:10.741619 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:10.743619 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.745049 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.746512 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.747129 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.747282 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:10.751875 (+ 4593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:10.751881 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:10.751894 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:10.751918 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:10.755147 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.755965 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.755972 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.755974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.756110 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.756430 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.756433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.756435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.757585 (+ 1150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:10.757921 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:10.758413 (+ 492us) server_negotiation.cc:299] Negotiation successful
0806 21:09:10.759068 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"spinlock_wait_cycles":60928,"thread_start_us":187,"threads_started":1}
I20260806 21:09:10.761586 9925 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 21:09:10 dist-test-slave-69rg krb5kdc[9687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050550, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.252@KRBTEST.COM for kudu/127.27.225.253@KRBTEST.COM
Aug 06 21:09:10 dist-test-slave-69rg krb5kdc[9687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:09:10.774389 9928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.762233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:46611 (local address 127.0.0.1:50910)
0806 21:09:10.762360 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.762375 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:10.762454 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:10.762865 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:10.762868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:10.763066 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:10.763217 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:10.763234 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.764306 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.764309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:10.764950 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:10.764960 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.765575 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.765579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:10.765690 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:10.766348 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:10.766364 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:10.768553 (+ 2189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:10.771181 (+ 2628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:10.771186 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:10.771190 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:10.772084 (+ 894us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:10.772578 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:10.772583 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:10.772585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:10.772718 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:10.773396 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:10.773401 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:10.773580 (+ 179us) client_negotiation.cc:769] Sending connection context
0806 21:09:10.773741 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 21:09:10.774172 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 21:09:10.774835 9854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.762390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50910 (local address 127.27.225.253:46611)
0806 21:09:10.762515 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:10.762520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:10.762533 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:09:10.762581 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:10.762585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:10.762632 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:10.762716 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:10.763419 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.764089 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.765168 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.765455 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.765536 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:10.768835 (+ 3299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:10.768839 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:10.768841 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:10.768870 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:10.771028 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.772315 (+ 1287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.772318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.772320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.772419 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.772844 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.772850 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.772852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.773167 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:10.773298 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:10.773888 (+ 590us) server_negotiation.cc:299] Negotiation successful
0806 21:09:10.774056 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 21:09:10.776247 9925 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:09:10.776753 9776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.763320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57154 (local address 127.27.225.254:37303)
0806 21:09:10.763440 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:10.763445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:10.763462 (+ 17us) server_negotiation.cc:407] Connection header received
0806 21:09:10.763594 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:10.763597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:10.763645 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:10.763739 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:10.764599 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.765238 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.766557 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.766817 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.766906 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:10.770931 (+ 4025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:10.770936 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:10.770939 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:10.770973 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:10.773871 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.774801 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.774804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.774805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.774905 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.775300 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.775304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.775305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.775585 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:10.776210 (+ 625us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:10.776338 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 21:09:10.776593 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 21:09:10 dist-test-slave-69rg krb5kdc[9687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050550, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.252@KRBTEST.COM for kudu/127.27.225.252@KRBTEST.COM
Aug 06 21:09:10 dist-test-slave-69rg krb5kdc[9687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.253@KRBTEST.COM
I20260806 21:09:10.788264 9928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.776909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:37579 (local address 127.0.0.1:53680)
0806 21:09:10.777307 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.777338 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:10.777572 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:10.778252 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:10.778255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:10.778431 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:10.778551 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:10.778559 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.779536 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.779542 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:10.780248 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:10.780257 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.781152 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.781156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:10.781242 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:10.781859 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:10.781875 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:10.783343 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:10.785750 (+ 2407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:10.785753 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:10.785755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:10.786158 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:10.786515 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:10.786517 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:10.786519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:10.786739 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:10.787598 (+ 859us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:10.787613 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:10.787749 (+ 136us) client_negotiation.cc:769] Sending connection context
0806 21:09:10.787881 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 21:09:10.788033 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238}
I20260806 21:09:10.788810 9929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.777572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53680 (local address 127.27.225.252:37579)
0806 21:09:10.777723 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:10.777726 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:10.777739 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:09:10.777771 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:10.777773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:10.777869 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:10.777960 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:10.778690 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.779379 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.780386 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.780991 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.781115 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:10.783596 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:10.783601 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:10.783608 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:10.783637 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:10.785578 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.786278 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.786281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.786282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.786359 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.786877 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.786879 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.786881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.787177 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:10.787279 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:10.787961 (+ 682us) server_negotiation.cc:299] Negotiation successful
0806 21:09:10.788571 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 21:09:10.790992 9854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.778644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50918 (local address 127.27.225.253:46611)
0806 21:09:10.778786 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:10.778790 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:10.778871 (+ 81us) server_negotiation.cc:407] Connection header received
0806 21:09:10.778919 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:10.778921 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:10.778971 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:10.779058 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:10.780049 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.780991 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.782099 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.782346 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.782588 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:10.786060 (+ 3472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:10.786066 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:10.786069 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:10.786106 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:10.788371 (+ 2265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.788887 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.788890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.788892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.788986 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.789357 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.789361 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.789362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.789904 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:10.790203 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:10.790414 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 21:09:10.790562 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 21:09:10.794088 9925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631: Bootstrap starting.
I20260806 21:09:10.796213 9771 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } attempt: 3
I20260806 21:09:10.800067 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:09:10.800490 9925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631: Neither blocks nor log segments found. Creating new log.
I20260806 21:09:10.801523 9925 log.cc:824] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631: Log is configured to *not* fsync() on all Append() calls
I20260806 21:09:10.805137 9925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631: No bootstrap required, opened a new log
Aug 06 21:09:10 dist-test-slave-69rg krb5kdc[9687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050550, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for kudu/127.27.225.252@KRBTEST.COM
I20260806 21:09:10.810245 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.798099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:37579 (local address 127.0.0.1:53684)
0806 21:09:10.798528 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.798547 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:10.798649 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:10.799587 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:10.799590 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:10.799848 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:10.799976 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:10.799986 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.800952 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.800956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:10.801494 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:10.801502 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.802192 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.802196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:10.802292 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:10.802878 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:10.802894 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:10.805551 (+ 2657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:10.808194 (+ 2643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:10.808198 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:10.808200 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:10.808469 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:10.808797 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:10.808801 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:10.808803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:10.808897 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:10.809581 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:10.809584 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:10.809702 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 21:09:10.809800 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 21:09:10.809994 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260806 21:09:10.810741 9925 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b31df38ec7724a53a55203ed9e131c84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37303 } } peers { permanent_uuid: "9ade41d137484cdfa993675c915c6ca6" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } } peers { permanent_uuid: "30c2a02670404bb1877f474ba1036631" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } }
I20260806 21:09:10.811106 9929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.798138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53684 (local address 127.27.225.252:37579)
0806 21:09:10.799341 (+ 1203us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:10.799346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:10.799359 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:09:10.799402 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:10.799405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:10.799451 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:10.799530 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:10.800089 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.800835 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.801663 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.802038 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.802155 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:10.806135 (+ 3980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:10.806141 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:10.806147 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:10.806178 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:10.808065 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.808594 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.808597 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.808599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.808692 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.809015 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.809018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.809020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.809438 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:10.810624 (+ 1186us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:10.810718 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 21:09:10.810958 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1114}
I20260806 21:09:10.811350 9925 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:09:10.811700 9925 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30c2a02670404bb1877f474ba1036631, State: Initialized, Role: FOLLOWER
I20260806 21:09:10.812770 9925 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b31df38ec7724a53a55203ed9e131c84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37303 } } peers { permanent_uuid: "9ade41d137484cdfa993675c915c6ca6" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } } peers { permanent_uuid: "30c2a02670404bb1877f474ba1036631" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } }
I20260806 21:09:10.815140 9925 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:09:10.815779 9776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.802802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57166 (local address 127.27.225.254:37303)
0806 21:09:10.802929 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:10.802933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:10.803829 (+ 896us) server_negotiation.cc:407] Connection header received
0806 21:09:10.804008 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:10.804011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:10.804056 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:10.804106 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:10.805099 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.805847 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.807112 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.807447 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.807534 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:10.809520 (+ 1986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:10.809524 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:10.809528 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:10.809553 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:10.812130 (+ 2577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.814004 (+ 1874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.814008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.814009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.814110 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.814664 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.814678 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.814680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.814925 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:10.815035 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:10.815446 (+ 411us) server_negotiation.cc:299] Negotiation successful
0806 21:09:10.815593 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 21:09:10.816174 9771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b31df38ec7724a53a55203ed9e131c84: Bootstrap starting.
I20260806 21:09:10.819859 9771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b31df38ec7724a53a55203ed9e131c84: Neither blocks nor log segments found. Creating new log.
I20260806 21:09:10.821045 9771 log.cc:824] T 00000000000000000000000000000000 P b31df38ec7724a53a55203ed9e131c84: Log is configured to *not* fsync() on all Append() calls
I20260806 21:09:10.822487 9854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.802916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50926 (local address 127.27.225.253:46611)
0806 21:09:10.803041 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:10.803048 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:10.804649 (+ 1601us) server_negotiation.cc:407] Connection header received
0806 21:09:10.805330 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:10.805333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:10.805392 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:10.805447 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:10.806936 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.807781 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.808994 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.809446 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.809549 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:10.813595 (+ 4046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:10.813599 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:10.813602 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:10.813636 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:10.816275 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.817262 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.817266 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.817267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.817371 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.821215 (+ 3844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.821220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.821225 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.821548 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:10.821714 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:10.822187 (+ 473us) server_negotiation.cc:299] Negotiation successful
0806 21:09:10.822330 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 21:09:10.822679 9951 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:09:10.822809 9951 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:09:10.823660 9771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b31df38ec7724a53a55203ed9e131c84: No bootstrap required, opened a new log
I20260806 21:09:10.820516 9856 master_runner.cc:186] Error getting master registration for 127.27.225.254:37303: OK, instance_id { permanent_uuid: "b31df38ec7724a53a55203ed9e131c84" instance_seqno: 1786050550155512 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 21:09:10.825160 9856 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 21:09:10.826073 9940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b31df38ec7724a53a55203ed9e131c84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37303 } } peers { permanent_uuid: "9ade41d137484cdfa993675c915c6ca6" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } } peers { permanent_uuid: "30c2a02670404bb1877f474ba1036631" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } } }
I20260806 21:09:10.826359 9940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:09:10.828107 9937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.803232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53694 (local address 127.27.225.252:37579)
0806 21:09:10.811102 (+ 7870us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:10.811111 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:10.811127 (+ 16us) server_negotiation.cc:407] Connection header received
0806 21:09:10.811262 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:10.811265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:10.811421 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:10.812673 (+ 1252us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:10.812799 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.817111 (+ 4312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.818773 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.819117 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.819219 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:10.823443 (+ 4224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:10.823447 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:10.823450 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:10.823479 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:10.825455 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.826100 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.826103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.826105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.826217 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.826590 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.826593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.826594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.826845 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:10.827690 (+ 845us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:10.827771 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 21:09:10.827908 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7796,"thread_start_us":89,"threads_started":1}
I20260806 21:09:10.829787 9771 raft_consensus.cc:348] T 00000000000000000000000000000000 P b31df38ec7724a53a55203ed9e131c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b31df38ec7724a53a55203ed9e131c84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37303 } } peers { permanent_uuid: "9ade41d137484cdfa993675c915c6ca6" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } } peers { permanent_uuid: "30c2a02670404bb1877f474ba1036631" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } }
I20260806 21:09:10.830116 9771 raft_consensus.cc:374] T 00000000000000000000000000000000 P b31df38ec7724a53a55203ed9e131c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:09:10.830214 9771 raft_consensus.cc:729] T 00000000000000000000000000000000 P b31df38ec7724a53a55203ed9e131c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b31df38ec7724a53a55203ed9e131c84, State: Initialized, Role: FOLLOWER
I20260806 21:09:10.830681 9771 consensus_queue.cc:260] T 00000000000000000000000000000000 P b31df38ec7724a53a55203ed9e131c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b31df38ec7724a53a55203ed9e131c84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37303 } } peers { permanent_uuid: "9ade41d137484cdfa993675c915c6ca6" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } } peers { permanent_uuid: "30c2a02670404bb1877f474ba1036631" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } }
I20260806 21:09:10.832983 9771 sys_catalog.cc:564] T 00000000000000000000000000000000 P b31df38ec7724a53a55203ed9e131c84 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:09:10.834120 9954 sys_catalog.cc:455] T 00000000000000000000000000000000 P b31df38ec7724a53a55203ed9e131c84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b31df38ec7724a53a55203ed9e131c84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37303 } } peers { permanent_uuid: "9ade41d137484cdfa993675c915c6ca6" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } } peers { permanent_uuid: "30c2a02670404bb1877f474ba1036631" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } } }
I20260806 21:09:10.834379 9954 sys_catalog.cc:458] T 00000000000000000000000000000000 P b31df38ec7724a53a55203ed9e131c84 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:09:10.839849 9849 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } attempt: 4
I20260806 21:09:10.840405 9702 master_runner.cc:186] Error getting master registration for 127.27.225.253:46611: OK, instance_id { permanent_uuid: "9ade41d137484cdfa993675c915c6ca6" instance_seqno: 1786050550420820 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
W20260806 21:09:10.845989 9966 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b31df38ec7724a53a55203ed9e131c84: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 21:09:10.846743 9702 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 21:09:10.846763 9966 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 06 21:09:10 dist-test-slave-69rg krb5kdc[9687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050550, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.253@KRBTEST.COM for kudu/127.27.225.252@KRBTEST.COM
I20260806 21:09:10.854677 9937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.840638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53696 (local address 127.27.225.252:37579)
0806 21:09:10.840908 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:10.840912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:10.840925 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:09:10.840964 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:10.840967 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:10.841010 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:10.841079 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:10.841891 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.842999 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.844194 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.844564 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.844691 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:10.849529 (+ 4838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:10.849534 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:10.849537 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:10.849573 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:10.852195 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.852891 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.852895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.852898 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.853013 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:10.853408 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:10.853412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:10.853413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:10.853675 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:10.853873 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:10.854371 (+ 498us) server_negotiation.cc:299] Negotiation successful
0806 21:09:10.854511 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188}
I20260806 21:09:10.855091 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:10.840342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:37579 (local address 127.0.0.1:53696)
0806 21:09:10.840651 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:10.840668 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:10.840754 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:10.841347 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:10.841351 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:10.841581 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:10.841694 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:10.841702 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.843144 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.843148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:10.844036 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:10.844051 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:10.844948 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:10.844953 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:10.845063 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:10.847102 (+ 2039us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:10.847119 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:10.849299 (+ 2180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:10.852382 (+ 3083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:10.852389 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:10.852392 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:10.852735 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:10.853145 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:10.853149 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:10.853151 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:10.853275 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:10.853878 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:10.853882 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:10.854019 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 21:09:10.854303 (+ 284us) client_negotiation.cc:240] Negotiation successful
0806 21:09:10.854499 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217}
I20260806 21:09:10.861322 9849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ade41d137484cdfa993675c915c6ca6: Bootstrap starting.
I20260806 21:09:10.864626 9849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ade41d137484cdfa993675c915c6ca6: Neither blocks nor log segments found. Creating new log.
I20260806 21:09:10.866529 9849 log.cc:824] T 00000000000000000000000000000000 P 9ade41d137484cdfa993675c915c6ca6: Log is configured to *not* fsync() on all Append() calls
I20260806 21:09:10.869983 9849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ade41d137484cdfa993675c915c6ca6: No bootstrap required, opened a new log
I20260806 21:09:10.877907 9849 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ade41d137484cdfa993675c915c6ca6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b31df38ec7724a53a55203ed9e131c84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37303 } } peers { permanent_uuid: "9ade41d137484cdfa993675c915c6ca6" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } } peers { permanent_uuid: "30c2a02670404bb1877f474ba1036631" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } }
I20260806 21:09:10.878814 9849 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ade41d137484cdfa993675c915c6ca6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:09:10.879051 9849 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ade41d137484cdfa993675c915c6ca6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ade41d137484cdfa993675c915c6ca6, State: Initialized, Role: FOLLOWER
I20260806 21:09:10.879781 9849 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ade41d137484cdfa993675c915c6ca6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b31df38ec7724a53a55203ed9e131c84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37303 } } peers { permanent_uuid: "9ade41d137484cdfa993675c915c6ca6" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } } peers { permanent_uuid: "30c2a02670404bb1877f474ba1036631" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } }
I20260806 21:09:10.880899 9969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ade41d137484cdfa993675c915c6ca6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b31df38ec7724a53a55203ed9e131c84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37303 } } peers { permanent_uuid: "9ade41d137484cdfa993675c915c6ca6" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } } peers { permanent_uuid: "30c2a02670404bb1877f474ba1036631" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } } }
I20260806 21:09:10.881124 9969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ade41d137484cdfa993675c915c6ca6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:09:10.882478 9849 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ade41d137484cdfa993675c915c6ca6 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 21:09:10.891080 9980 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9ade41d137484cdfa993675c915c6ca6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:09:10.891336 9980 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:09:10.894330 9779 master_runner.cc:263] No leader master found from master 9ade41d137484cdfa993675c915c6ca6
I20260806 21:09:10.894428 9779 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 21:09:10.901721 9724 tablet.cc:2369] T 00000000000000000000000000000000 P b31df38ec7724a53a55203ed9e131c84: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 21:09:10.916898 9800 tablet.cc:2369] T 00000000000000000000000000000000 P 9ade41d137484cdfa993675c915c6ca6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 21:09:10.968353 9877 tablet.cc:2369] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:09:11.022868 9940 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:09:11.023139 9940 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b31df38ec7724a53a55203ed9e131c84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37303 } } peers { permanent_uuid: "9ade41d137484cdfa993675c915c6ca6" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } } peers { permanent_uuid: "30c2a02670404bb1877f474ba1036631" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } }
I20260806 21:09:11.024458 9940 leader_election.cc:288] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b31df38ec7724a53a55203ed9e131c84 (127.27.225.254:37303), 9ade41d137484cdfa993675c915c6ca6 (127.27.225.253:46611)
I20260806 21:09:11.025104 9745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30c2a02670404bb1877f474ba1036631" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b31df38ec7724a53a55203ed9e131c84" is_pre_election: true
I20260806 21:09:11.025486 9745 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b31df38ec7724a53a55203ed9e131c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30c2a02670404bb1877f474ba1036631 in term 0.
I20260806 21:09:11.026639 9821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30c2a02670404bb1877f474ba1036631" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ade41d137484cdfa993675c915c6ca6" is_pre_election: true
I20260806 21:09:11.027009 9821 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9ade41d137484cdfa993675c915c6ca6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30c2a02670404bb1877f474ba1036631 in term 0.
I20260806 21:09:11.028007 9873 leader_election.cc:302] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30c2a02670404bb1877f474ba1036631, 9ade41d137484cdfa993675c915c6ca6; no voters:
I20260806 21:09:11.028471 9940 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:09:11.028568 9940 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:09:11.028615 9940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:09:11.030795 9940 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b31df38ec7724a53a55203ed9e131c84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37303 } } peers { permanent_uuid: "9ade41d137484cdfa993675c915c6ca6" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } } peers { permanent_uuid: "30c2a02670404bb1877f474ba1036631" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } }
I20260806 21:09:11.033031 9821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30c2a02670404bb1877f474ba1036631" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ade41d137484cdfa993675c915c6ca6"
I20260806 21:09:11.033228 9821 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ade41d137484cdfa993675c915c6ca6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:09:11.036082 9821 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9ade41d137484cdfa993675c915c6ca6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30c2a02670404bb1877f474ba1036631 in term 1.
I20260806 21:09:11.038300 9873 leader_election.cc:302] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30c2a02670404bb1877f474ba1036631, 9ade41d137484cdfa993675c915c6ca6; no voters:
I20260806 21:09:11.039575 9745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30c2a02670404bb1877f474ba1036631" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b31df38ec7724a53a55203ed9e131c84"
I20260806 21:09:11.039793 9745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b31df38ec7724a53a55203ed9e131c84 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:09:11.041390 9745 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b31df38ec7724a53a55203ed9e131c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30c2a02670404bb1877f474ba1036631 in term 1.
I20260806 21:09:11.042601 9940 leader_election.cc:288] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [CANDIDATE]: Term 1 election: Requested vote from peers b31df38ec7724a53a55203ed9e131c84 (127.27.225.254:37303), 9ade41d137484cdfa993675c915c6ca6 (127.27.225.253:46611)
I20260806 21:09:11.042906 9940 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:09:11.043118 9940 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [term 1 LEADER]: Becoming Leader. State: Replica: 30c2a02670404bb1877f474ba1036631, State: Running, Role: LEADER
I20260806 21:09:11.043588 9940 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b31df38ec7724a53a55203ed9e131c84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37303 } } peers { permanent_uuid: "9ade41d137484cdfa993675c915c6ca6" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } } peers { permanent_uuid: "30c2a02670404bb1877f474ba1036631" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } }
I20260806 21:09:11.046542 9940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30c2a02670404bb1877f474ba1036631. Latest consensus state: current_term: 1 leader_uuid: "30c2a02670404bb1877f474ba1036631" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b31df38ec7724a53a55203ed9e131c84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37303 } } peers { permanent_uuid: "9ade41d137484cdfa993675c915c6ca6" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } } peers { permanent_uuid: "30c2a02670404bb1877f474ba1036631" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } } }
I20260806 21:09:11.052284 9940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [sys.catalog]: This master's current role is: LEADER
I20260806 21:09:11.059873 9984 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:09:11.060904 9984 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:09:11.081202 9821 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9ade41d137484cdfa993675c915c6ca6 [term 1 FOLLOWER]: Refusing update from remote peer 30c2a02670404bb1877f474ba1036631: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:09:11.082098 9940 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ade41d137484cdfa993675c915c6ca6" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:09:11.087173 9745 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b31df38ec7724a53a55203ed9e131c84 [term 1 FOLLOWER]: Refusing update from remote peer 30c2a02670404bb1877f474ba1036631: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:09:11.087919 9940 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b31df38ec7724a53a55203ed9e131c84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:09:11.088207 9969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ade41d137484cdfa993675c915c6ca6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30c2a02670404bb1877f474ba1036631. Latest consensus state: current_term: 1 leader_uuid: "30c2a02670404bb1877f474ba1036631" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b31df38ec7724a53a55203ed9e131c84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37303 } } peers { permanent_uuid: "9ade41d137484cdfa993675c915c6ca6" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } } peers { permanent_uuid: "30c2a02670404bb1877f474ba1036631" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } } }
I20260806 21:09:11.088384 9969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ade41d137484cdfa993675c915c6ca6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:09:11.093650 9954 sys_catalog.cc:455] T 00000000000000000000000000000000 P b31df38ec7724a53a55203ed9e131c84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30c2a02670404bb1877f474ba1036631. Latest consensus state: current_term: 1 leader_uuid: "30c2a02670404bb1877f474ba1036631" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b31df38ec7724a53a55203ed9e131c84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37303 } } peers { permanent_uuid: "9ade41d137484cdfa993675c915c6ca6" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } } peers { permanent_uuid: "30c2a02670404bb1877f474ba1036631" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } } }
I20260806 21:09:11.093844 9954 sys_catalog.cc:458] T 00000000000000000000000000000000 P b31df38ec7724a53a55203ed9e131c84 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:09:11.104141 9940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "30c2a02670404bb1877f474ba1036631" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b31df38ec7724a53a55203ed9e131c84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37303 } } peers { permanent_uuid: "9ade41d137484cdfa993675c915c6ca6" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } } peers { permanent_uuid: "30c2a02670404bb1877f474ba1036631" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } } }
I20260806 21:09:11.104348 9940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [sys.catalog]: This master's current role is: LEADER
I20260806 21:09:11.106077 9954 sys_catalog.cc:455] T 00000000000000000000000000000000 P b31df38ec7724a53a55203ed9e131c84 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "30c2a02670404bb1877f474ba1036631" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b31df38ec7724a53a55203ed9e131c84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37303 } } peers { permanent_uuid: "9ade41d137484cdfa993675c915c6ca6" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } } peers { permanent_uuid: "30c2a02670404bb1877f474ba1036631" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } } }
I20260806 21:09:11.106725 9954 sys_catalog.cc:458] T 00000000000000000000000000000000 P b31df38ec7724a53a55203ed9e131c84 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:09:11.107172 9984 catalog_manager.cc:1347] Generated new cluster ID: e6beaf19604e4cdd95a06349835ea71a
I20260806 21:09:11.107319 9984 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:09:11.117534 9969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ade41d137484cdfa993675c915c6ca6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "30c2a02670404bb1877f474ba1036631" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b31df38ec7724a53a55203ed9e131c84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37303 } } peers { permanent_uuid: "9ade41d137484cdfa993675c915c6ca6" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } } peers { permanent_uuid: "30c2a02670404bb1877f474ba1036631" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } } }
I20260806 21:09:11.117972 9969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ade41d137484cdfa993675c915c6ca6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:09:11.118321 9982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "30c2a02670404bb1877f474ba1036631" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b31df38ec7724a53a55203ed9e131c84" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 37303 } } peers { permanent_uuid: "9ade41d137484cdfa993675c915c6ca6" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 46611 } } peers { permanent_uuid: "30c2a02670404bb1877f474ba1036631" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 37579 } } }
I20260806 21:09:11.118489 9982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631 [sys.catalog]: This master's current role is: LEADER
I20260806 21:09:11.119048 9986 mvcc.cc:205] Tried to move back new op lower bound from 7315663057214926848 to 7315663057080725504. Current Snapshot: MvccSnapshot[applied={T|T < 7315663057214926848}]
I20260806 21:09:11.124392 9988 mvcc.cc:205] Tried to move back new op lower bound from 7315663057214926848 to 7315663057080725504. Current Snapshot: MvccSnapshot[applied={T|T < 7315663057214926848}]
I20260806 21:09:11.156847 9984 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:09:11.158397 9984 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:09:11.183120 9984 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30c2a02670404bb1877f474ba1036631: Generated new TSK 0
I20260806 21:09:11.184208 9984 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:09:11.848662 9966 catalog_manager.cc:1259] Loaded cluster ID: e6beaf19604e4cdd95a06349835ea71a
I20260806 21:09:11.848770 9966 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b31df38ec7724a53a55203ed9e131c84: loading cluster ID for follower catalog manager: success
I20260806 21:09:11.850914 9966 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b31df38ec7724a53a55203ed9e131c84: acquiring CA information for follower catalog manager: success
I20260806 21:09:11.852259 9966 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b31df38ec7724a53a55203ed9e131c84: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 21:09:11.892789 9980 catalog_manager.cc:1259] Loaded cluster ID: e6beaf19604e4cdd95a06349835ea71a
I20260806 21:09:11.892884 9980 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9ade41d137484cdfa993675c915c6ca6: loading cluster ID for follower catalog manager: success
I20260806 21:09:11.895095 9980 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9ade41d137484cdfa993675c915c6ca6: acquiring CA information for follower catalog manager: success
I20260806 21:09:11.896832 9980 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9ade41d137484cdfa993675c915c6ca6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 21:09:12.267241 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 9702
I20260806 21:09:12.276034 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 9779
I20260806 21:09:12.283281 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 9856
2026-08-06T21:09:12Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3666 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:09:12 dist-test-slave-69rg krb5kdc[10001](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:09:12 dist-test-slave-69rg krb5kdc[10001](info): set up 2 sockets
Aug 06 21:09:12 dist-test-slave-69rg krb5kdc[10001](info): commencing operation
krb5kdc: starting...
W20260806 21:09:13.474390 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.007s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:09:13 dist-test-slave-69rg krb5kdc[10001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:09:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:09:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:09:13.626420 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:33615
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:45883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:33615
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:09:13.765350 10015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:13.765779 10015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:13.765909 10015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:13.770861 10015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:09:13.771052 10015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:13.771095 10015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:13.771130 10015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:09:13.771169 10015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:09:13.781498 10015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:45883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:33615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:33615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:13.783201 10015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:13.784430 10015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:13.794866 10021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:13.794869 10023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:13.795161 10020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:13.795286 10015 server_base.cc:1034] running on GCE node
I20260806 21:09:13.796622 10015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:13.798032 10015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:13.799328 10015 hybrid_clock.cc:648] HybridClock initialized: now 1786050553799299 us; error 56 us; skew 500 ppm
Aug 06 21:09:13 dist-test-slave-69rg krb5kdc[10001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050553, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:13.802922 10015 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:09:13.803954 10015 webserver.cc:460] Webserver started at http://127.27.225.254:42743/ using document root <none> and password file <none>
I20260806 21:09:13.804577 10015 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:13.804682 10015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:13.804901 10015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:09:13.807253 10015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "e4f83b17783544bdb3a7c3dec116d238"
format_stamp: "Formatted at 2026-08-06 21:09:13 on dist-test-slave-69rg"
server_key: "5faee654c44d58d251431914a8747306"
server_key_iv: "22a5546dd1769f00b1935c9098f56e19"
server_key_version: "encryptionkey@0"
I20260806 21:09:13.807875 10015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "e4f83b17783544bdb3a7c3dec116d238"
format_stamp: "Formatted at 2026-08-06 21:09:13 on dist-test-slave-69rg"
server_key: "5faee654c44d58d251431914a8747306"
server_key_iv: "22a5546dd1769f00b1935c9098f56e19"
server_key_version: "encryptionkey@0"
I20260806 21:09:13.812434 10015 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:09:13.815873 10030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:13.817530 10015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:09:13.817723 10015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "e4f83b17783544bdb3a7c3dec116d238"
format_stamp: "Formatted at 2026-08-06 21:09:13 on dist-test-slave-69rg"
server_key: "5faee654c44d58d251431914a8747306"
server_key_iv: "22a5546dd1769f00b1935c9098f56e19"
server_key_version: "encryptionkey@0"
I20260806 21:09:13.817946 10015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:13.854919 10015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:13.860706 10015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:13.860906 10015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:13.868784 10015 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:33615
I20260806 21:09:13.868863 10084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:33615 every 8 connection(s)
I20260806 21:09:13.869884 10015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:09:13.872974 10085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:09:13.876807 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 10015
I20260806 21:09:13.876930 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:09:13.877277 28551 external_mini_cluster.cc:1428] Setting key 7584cc7eee6772f87b69333e825e592c
I20260806 21:09:13.879742 10085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4f83b17783544bdb3a7c3dec116d238: Bootstrap starting.
I20260806 21:09:13.883747 10085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4f83b17783544bdb3a7c3dec116d238: Neither blocks nor log segments found. Creating new log.
I20260806 21:09:13.884749 10085 log.cc:824] T 00000000000000000000000000000000 P e4f83b17783544bdb3a7c3dec116d238: Log is configured to *not* fsync() on all Append() calls
I20260806 21:09:13.886967 10085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4f83b17783544bdb3a7c3dec116d238: No bootstrap required, opened a new log
Aug 06 21:09:13 dist-test-slave-69rg krb5kdc[10001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:09:13.892957 10085 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4f83b17783544bdb3a7c3dec116d238 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4f83b17783544bdb3a7c3dec116d238" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 33615 } }
I20260806 21:09:13.893260 10085 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4f83b17783544bdb3a7c3dec116d238 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:09:13.893363 10085 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4f83b17783544bdb3a7c3dec116d238 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f83b17783544bdb3a7c3dec116d238, State: Initialized, Role: FOLLOWER
I20260806 21:09:13.893994 10085 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4f83b17783544bdb3a7c3dec116d238 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4f83b17783544bdb3a7c3dec116d238" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 33615 } }
I20260806 21:09:13.894141 10085 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4f83b17783544bdb3a7c3dec116d238 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:09:13.894234 10085 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4f83b17783544bdb3a7c3dec116d238 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:09:13.894371 10085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4f83b17783544bdb3a7c3dec116d238 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:09:13.895670 10085 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4f83b17783544bdb3a7c3dec116d238 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4f83b17783544bdb3a7c3dec116d238" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 33615 } }
I20260806 21:09:13.896062 10085 leader_election.cc:302] T 00000000000000000000000000000000 P e4f83b17783544bdb3a7c3dec116d238 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4f83b17783544bdb3a7c3dec116d238; no voters:
I20260806 21:09:13.896450 10085 leader_election.cc:288] T 00000000000000000000000000000000 P e4f83b17783544bdb3a7c3dec116d238 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:09:13.896548 10090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4f83b17783544bdb3a7c3dec116d238 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:09:13.896731 10090 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4f83b17783544bdb3a7c3dec116d238 [term 1 LEADER]: Becoming Leader. State: Replica: e4f83b17783544bdb3a7c3dec116d238, State: Running, Role: LEADER
I20260806 21:09:13.896781 10088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:13.878843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55602 (local address 127.27.225.254:33615)
0806 21:09:13.879309 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:13.879329 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:13.879371 (+ 42us) server_negotiation.cc:407] Connection header received
0806 21:09:13.880289 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:13.880314 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:13.880860 (+ 546us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:13.881361 (+ 501us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:13.882354 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:13.884401 (+ 2047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:13.885451 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:13.886029 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:13.886217 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:13.889373 (+ 3156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:13.889379 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:13.889477 (+ 98us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:13.889513 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:13.892098 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:13.892829 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:13.892837 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:13.892843 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:13.892995 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:13.893382 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:13.893387 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:13.893389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:13.894151 (+ 762us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:13.894414 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:13.895031 (+ 617us) server_negotiation.cc:299] Negotiation successful
0806 21:09:13.895490 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":138,"threads_started":1}
I20260806 21:09:13.897080 10090 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4f83b17783544bdb3a7c3dec116d238 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4f83b17783544bdb3a7c3dec116d238" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 33615 } }
I20260806 21:09:13.897913 10085 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4f83b17783544bdb3a7c3dec116d238 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:09:13.898391 10092 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4f83b17783544bdb3a7c3dec116d238 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4f83b17783544bdb3a7c3dec116d238" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4f83b17783544bdb3a7c3dec116d238" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 33615 } } }
I20260806 21:09:13.898564 10092 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4f83b17783544bdb3a7c3dec116d238 [sys.catalog]: This master's current role is: LEADER
I20260806 21:09:13.898371 10091 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4f83b17783544bdb3a7c3dec116d238 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4f83b17783544bdb3a7c3dec116d238. Latest consensus state: current_term: 1 leader_uuid: "e4f83b17783544bdb3a7c3dec116d238" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4f83b17783544bdb3a7c3dec116d238" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 33615 } } }
I20260806 21:09:13.898947 10091 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4f83b17783544bdb3a7c3dec116d238 [sys.catalog]: This master's current role is: LEADER
I20260806 21:09:13.900806 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 21:09:13.903851 10107 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e4f83b17783544bdb3a7c3dec116d238: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:09:13.903956 10107 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:09:13.904044 10095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:09:13.904837 10095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:09:13.912302 10095 catalog_manager.cc:1347] Generated new cluster ID: a8c3caf137db42b9baa73953d9dfe6c1
I20260806 21:09:13.912405 10095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:09:13.931945 10095 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:09:13.933341 10095 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:09:13.943171 10095 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4f83b17783544bdb3a7c3dec116d238: Generated new TSK 0
I20260806 21:09:13.944069 10095 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:09:13.957540 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 10015
2026-08-06T21:09:13Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1670 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:09:14 dist-test-slave-69rg krb5kdc[10114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:09:14 dist-test-slave-69rg krb5kdc[10114](info): set up 2 sockets
Aug 06 21:09:14 dist-test-slave-69rg krb5kdc[10114](info): commencing operation
krb5kdc: starting...
W20260806 21:09:15.132383 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:09:15 dist-test-slave-69rg krb5kdc[10114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:09:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:09:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:09:15.307677 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:40157
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:36065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:40157
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:09:15.441962 10127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:15.442327 10127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:15.442416 10127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:15.447805 10127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:09:15.447929 10127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:15.447968 10127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:15.448012 10127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:09:15.448050 10127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:09:15.454072 10127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:36065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:40157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:40157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:15.455330 10127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:15.456441 10127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:15.466202 10135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:15.466235 10132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:15.466392 10127 server_base.cc:1034] running on GCE node
W20260806 21:09:15.466202 10133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:15.467293 10127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:15.468714 10127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:15.469903 10127 hybrid_clock.cc:648] HybridClock initialized: now 1786050555469886 us; error 36 us; skew 500 ppm
Aug 06 21:09:15 dist-test-slave-69rg krb5kdc[10114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050555, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:15.472892 10127 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:09:15.473973 10127 webserver.cc:460] Webserver started at http://127.27.225.254:37955/ using document root <none> and password file <none>
I20260806 21:09:15.474514 10127 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:15.474577 10127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:15.474761 10127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:09:15.476840 10127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "a8326cac2b564a289b3a80bca33e7d38"
format_stamp: "Formatted at 2026-08-06 21:09:15 on dist-test-slave-69rg"
server_key: "1f1319a8ecda1ae6596ac9cad82df334"
server_key_iv: "d33404251e8182f753e8bb88219dcb48"
server_key_version: "encryptionkey@0"
I20260806 21:09:15.477375 10127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "a8326cac2b564a289b3a80bca33e7d38"
format_stamp: "Formatted at 2026-08-06 21:09:15 on dist-test-slave-69rg"
server_key: "1f1319a8ecda1ae6596ac9cad82df334"
server_key_iv: "d33404251e8182f753e8bb88219dcb48"
server_key_version: "encryptionkey@0"
I20260806 21:09:15.481215 10127 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260806 21:09:15.483875 10142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:15.485352 10127 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 21:09:15.485472 10127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "a8326cac2b564a289b3a80bca33e7d38"
format_stamp: "Formatted at 2026-08-06 21:09:15 on dist-test-slave-69rg"
server_key: "1f1319a8ecda1ae6596ac9cad82df334"
server_key_iv: "d33404251e8182f753e8bb88219dcb48"
server_key_version: "encryptionkey@0"
I20260806 21:09:15.485579 10127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:15.512588 10127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:15.516063 10127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:15.516245 10127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:15.524241 10127 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:40157
I20260806 21:09:15.524276 10195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:40157 every 8 connection(s)
I20260806 21:09:15.525264 10127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:09:15.527256 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 10127
I20260806 21:09:15.527391 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:09:15.527693 28551 external_mini_cluster.cc:1428] Setting key 35393382c6f030cc7340e3e0f207d91e
I20260806 21:09:15.528407 10196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:09:15.534422 10196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38: Bootstrap starting.
Aug 06 21:09:15 dist-test-slave-69rg krb5kdc[10114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:09:15.537133 10196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38: Neither blocks nor log segments found. Creating new log.
I20260806 21:09:15.538039 10196 log.cc:824] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38: Log is configured to *not* fsync() on all Append() calls
I20260806 21:09:15.540189 10196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38: No bootstrap required, opened a new log
I20260806 21:09:15.542413 10199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:15.529152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51720 (local address 127.27.225.254:40157)
0806 21:09:15.529628 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:15.529642 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:15.529681 (+ 39us) server_negotiation.cc:407] Connection header received
0806 21:09:15.530406 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:15.530422 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:15.530736 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:15.531080 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:15.531924 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:15.532930 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:15.533802 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:15.534340 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:15.534516 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:15.537395 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:15.537400 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:15.537415 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:15.537447 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:15.539415 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:15.539911 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:15.539924 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:15.539928 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:15.540043 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:15.540362 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:15.540365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:15.540367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:15.540789 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:15.541001 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:15.541302 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 21:09:15.541606 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":142,"threads_started":1}
I20260806 21:09:15.544378 10196 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8326cac2b564a289b3a80bca33e7d38" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40157 } }
I20260806 21:09:15.544696 10196 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:09:15.544816 10196 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8326cac2b564a289b3a80bca33e7d38, State: Initialized, Role: FOLLOWER
I20260806 21:09:15.545303 10196 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8326cac2b564a289b3a80bca33e7d38" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40157 } }
I20260806 21:09:15.545473 10196 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:09:15.545565 10196 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:09:15.545691 10196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:09:15.546790 10196 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8326cac2b564a289b3a80bca33e7d38" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40157 } }
I20260806 21:09:15.547185 10196 leader_election.cc:302] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8326cac2b564a289b3a80bca33e7d38; no voters:
I20260806 21:09:15.547567 10196 leader_election.cc:288] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:09:15.547689 10201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:09:15.547920 10201 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [term 1 LEADER]: Becoming Leader. State: Replica: a8326cac2b564a289b3a80bca33e7d38, State: Running, Role: LEADER
I20260806 21:09:15.548277 10201 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8326cac2b564a289b3a80bca33e7d38" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40157 } }
I20260806 21:09:15.548883 10196 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:09:15.550050 10203 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8326cac2b564a289b3a80bca33e7d38. Latest consensus state: current_term: 1 leader_uuid: "a8326cac2b564a289b3a80bca33e7d38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8326cac2b564a289b3a80bca33e7d38" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40157 } } }
I20260806 21:09:15.550122 10202 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8326cac2b564a289b3a80bca33e7d38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8326cac2b564a289b3a80bca33e7d38" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40157 } } }
I20260806 21:09:15.550202 10203 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [sys.catalog]: This master's current role is: LEADER
I20260806 21:09:15.550230 10202 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [sys.catalog]: This master's current role is: LEADER
I20260806 21:09:15.550599 10210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:09:15.553614 10210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:09:15.559772 10210 catalog_manager.cc:1347] Generated new cluster ID: f62d58e3dd3441cd86ccc5ffed368f7a
I20260806 21:09:15.559872 10210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:09:15.578332 10210 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:09:15.579387 10210 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:09:15.588706 10210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38: Generated new TSK 0
I20260806 21:09:15.589628 10210 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.225.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
I20260806 21:09:15.631706 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40157
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:36065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:09:15.753075 10223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:15.753453 10223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:15.753537 10223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:15.757930 10223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:15.758055 10223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:15.758175 10223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:09:15.763731 10223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:36065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:15.764887 10223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:15.765954 10223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:15.775760 10231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:15.775760 10228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:15.775808 10229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:15.776175 10223 server_base.cc:1034] running on GCE node
I20260806 21:09:15.776656 10223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:15.777388 10223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:15.778604 10223 hybrid_clock.cc:648] HybridClock initialized: now 1786050555778571 us; error 49 us; skew 500 ppm
Aug 06 21:09:15 dist-test-slave-69rg krb5kdc[10114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050555, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:15.782096 10223 init.cc:377] Logged in from keytab as kudu/127.27.225.193@KRBTEST.COM (short username kudu)
I20260806 21:09:15.783123 10223 webserver.cc:460] Webserver started at http://127.27.225.193:35679/ using document root <none> and password file <none>
I20260806 21:09:15.783797 10223 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:15.783905 10223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:15.784157 10223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:09:15.786540 10223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "04cf9ddb3b194f778e793f21766db37d"
format_stamp: "Formatted at 2026-08-06 21:09:15 on dist-test-slave-69rg"
server_key: "424bbe2cb4e5e30ee84733d739b134a5"
server_key_iv: "b73fec191d17567b21271906298cc16a"
server_key_version: "encryptionkey@0"
I20260806 21:09:15.787192 10223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "04cf9ddb3b194f778e793f21766db37d"
format_stamp: "Formatted at 2026-08-06 21:09:15 on dist-test-slave-69rg"
server_key: "424bbe2cb4e5e30ee84733d739b134a5"
server_key_iv: "b73fec191d17567b21271906298cc16a"
server_key_version: "encryptionkey@0"
I20260806 21:09:15.791733 10223 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:09:15.794530 10238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:15.796018 10223 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:09:15.796196 10223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "04cf9ddb3b194f778e793f21766db37d"
format_stamp: "Formatted at 2026-08-06 21:09:15 on dist-test-slave-69rg"
server_key: "424bbe2cb4e5e30ee84733d739b134a5"
server_key_iv: "b73fec191d17567b21271906298cc16a"
server_key_version: "encryptionkey@0"
I20260806 21:09:15.796356 10223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:15.814615 10223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:15.818608 10223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:15.818845 10223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:15.819510 10223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:09:15.820479 10223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:09:15.820554 10223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:15.820627 10223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:09:15.820688 10223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:15.830185 10223 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:35613
I20260806 21:09:15.830215 10355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:35613 every 8 connection(s)
I20260806 21:09:15.831403 10223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:09:15.840484 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 10223
I20260806 21:09:15.840857 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:09:15.841383 28551 external_mini_cluster.cc:1428] Setting key 686194069ecfc924c26d19fd139b1e8f
Aug 06 21:09:15 dist-test-slave-69rg krb5kdc[10114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050555, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:09:15.853032 10199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:15.833843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:52331 (local address 127.27.225.254:40157)
0806 21:09:15.834005 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:15.834011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:15.835110 (+ 1099us) server_negotiation.cc:407] Connection header received
0806 21:09:15.836758 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:15.836771 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:15.836840 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:15.836918 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:15.839242 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:15.839948 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:15.841902 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:15.842173 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:15.842275 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:15.846548 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:15.846554 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:15.846558 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:15.846602 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:15.848736 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:15.850075 (+ 1339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:15.850083 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:15.850084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:15.850281 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:15.851082 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:15.851086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:15.851088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:15.851636 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:15.851800 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:15.852445 (+ 645us) server_negotiation.cc:299] Negotiation successful
0806 21:09:15.852838 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 21:09:15.854283 10358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:15.834282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.193:52331)
0806 21:09:15.834931 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:15.834980 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:15.836415 (+ 1435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:15.837250 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:15.837258 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:15.837670 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:15.838677 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:15.838706 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:15.840251 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:15.840255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:15.841649 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:15.841667 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:15.842448 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:15.842454 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:15.842846 (+ 392us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:15.844118 (+ 1272us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:15.844140 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:15.846282 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:15.848912 (+ 2630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:15.848924 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:15.848950 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:15.849426 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:15.850426 (+ 1000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:15.850430 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:15.850432 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:15.850947 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:15.851801 (+ 854us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:15.851811 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:15.852149 (+ 338us) client_negotiation.cc:769] Sending connection context
0806 21:09:15.852381 (+ 232us) client_negotiation.cc:240] Negotiation successful
0806 21:09:15.853035 (+ 654us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":151,"threads_started":1}
WARNING: no policy specified for kudu/127.27.225.194@KRBTEST.COM; defaulting to no policy
I20260806 21:09:15.856300 10356 heartbeater.cc:347] Connected to a master server at 127.27.225.254:40157
I20260806 21:09:15.856698 10356 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:15.857364 10356 heartbeater.cc:511] Master 127.27.225.254:40157 requested a full tablet report, sending...
I20260806 21:09:15.859483 10160 ts_manager.cc:194] Registered new tserver with Master: 04cf9ddb3b194f778e793f21766db37d (127.27.225.193:35613)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.194@KRBTEST.COM" created.
I20260806 21:09:15.860955 10160 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.193@KRBTEST.COM'} at 127.27.225.193:52331
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
I20260806 21:09:15.887432 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40157
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:36065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:09:16.029627 10361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:16.030004 10361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:16.030053 10361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:16.034924 10361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:16.035337 10361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:16.035734 10361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:09:16.043186 10361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:36065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:16.044433 10361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:16.045521 10361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:16.058868 10366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:16.058917 10370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:16.059099 10367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:16.059401 10361 server_base.cc:1034] running on GCE node
I20260806 21:09:16.060509 10361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:16.061479 10361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:16.064982 10361 hybrid_clock.cc:648] HybridClock initialized: now 1786050556064941 us; error 89 us; skew 500 ppm
Aug 06 21:09:16 dist-test-slave-69rg krb5kdc[10114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050556, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:16.074251 10361 init.cc:377] Logged in from keytab as kudu/127.27.225.194@KRBTEST.COM (short username kudu)
I20260806 21:09:16.075366 10361 webserver.cc:460] Webserver started at http://127.27.225.194:38499/ using document root <none> and password file <none>
I20260806 21:09:16.076102 10361 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:16.076186 10361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:16.076476 10361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:09:16.080019 10361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "40237e8dc68b4fc6ada088cfba203a52"
format_stamp: "Formatted at 2026-08-06 21:09:16 on dist-test-slave-69rg"
server_key: "ea5909bb6f9227fdd8115e213b96bb99"
server_key_iv: "c06b568a59b8c2d4c779ef16b9d924fe"
server_key_version: "encryptionkey@0"
I20260806 21:09:16.080897 10361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "40237e8dc68b4fc6ada088cfba203a52"
format_stamp: "Formatted at 2026-08-06 21:09:16 on dist-test-slave-69rg"
server_key: "ea5909bb6f9227fdd8115e213b96bb99"
server_key_iv: "c06b568a59b8c2d4c779ef16b9d924fe"
server_key_version: "encryptionkey@0"
I20260806 21:09:16.086177 10361 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 21:09:16.089354 10377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:16.091096 10361 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:09:16.091243 10361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "40237e8dc68b4fc6ada088cfba203a52"
format_stamp: "Formatted at 2026-08-06 21:09:16 on dist-test-slave-69rg"
server_key: "ea5909bb6f9227fdd8115e213b96bb99"
server_key_iv: "c06b568a59b8c2d4c779ef16b9d924fe"
server_key_version: "encryptionkey@0"
I20260806 21:09:16.091384 10361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:16.107110 10361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:16.110828 10361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:16.111089 10361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:16.111821 10361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:09:16.112762 10361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:09:16.112841 10361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:16.112915 10361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:09:16.112962 10361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:16.125437 10361 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:43135
I20260806 21:09:16.125460 10495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:43135 every 8 connection(s)
I20260806 21:09:16.126470 10361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:09:16.127789 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 10361
I20260806 21:09:16.127930 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:09:16.128237 28551 external_mini_cluster.cc:1428] Setting key c073239145b80dd7f23b740b11bc91b3
Aug 06 21:09:16 dist-test-slave-69rg krb5kdc[10114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050556, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.225.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.195@KRBTEST.COM" created.
I20260806 21:09:16.144701 10199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:16.128740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:53243 (local address 127.27.225.254:40157)
0806 21:09:16.128909 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:16.128913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:16.130008 (+ 1095us) server_negotiation.cc:407] Connection header received
0806 21:09:16.132111 (+ 2103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:16.132114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:16.132197 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:16.132321 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:16.133557 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.134173 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.136234 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.136719 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.136885 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:16.140261 (+ 3376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:16.140267 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:16.140270 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:16.140301 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:16.142272 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:16.142921 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:16.142924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:16.142925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:16.143014 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:16.143383 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:16.143385 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:16.143387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:16.143675 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:16.143794 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:16.144377 (+ 583us) server_negotiation.cc:299] Negotiation successful
0806 21:09:16.144504 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 21:09:16.145671 10498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:16.129175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.194:53243)
0806 21:09:16.129793 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:16.129891 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:16.131858 (+ 1967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:16.132505 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:16.132513 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:16.132902 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:16.133393 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:16.133407 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.134323 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.134329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:16.136050 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:16.136067 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.136923 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.136927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:16.137104 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:16.138139 (+ 1035us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:16.138170 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:16.140031 (+ 1861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:16.142439 (+ 2408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:16.142450 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:16.142472 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:16.142802 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:16.143110 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:16.143112 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:16.143114 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:16.143258 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:16.143835 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:16.143844 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:16.144125 (+ 281us) client_negotiation.cc:769] Sending connection context
0806 21:09:16.144311 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 21:09:16.144713 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":168,"threads_started":1}
I20260806 21:09:16.147042 10496 heartbeater.cc:347] Connected to a master server at 127.27.225.254:40157
I20260806 21:09:16.147300 10496 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:16.147861 10496 heartbeater.cc:511] Master 127.27.225.254:40157 requested a full tablet report, sending...
I20260806 21:09:16.149046 10160 ts_manager.cc:194] Registered new tserver with Master: 40237e8dc68b4fc6ada088cfba203a52 (127.27.225.194:43135)
I20260806 21:09:16.149792 10160 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.194@KRBTEST.COM'} at 127.27.225.194:53243
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
I20260806 21:09:16.174962 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40157
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:36065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:09:16.301270 10501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:16.301595 10501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:16.301646 10501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:16.306463 10501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:16.306552 10501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:16.306622 10501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:09:16.312930 10501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:36065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:40157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:16.314271 10501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:16.315408 10501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:16.326910 10509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:16.327168 10507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:16.327258 10506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:16.327390 10501 server_base.cc:1034] running on GCE node
I20260806 21:09:16.328485 10501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:16.329216 10501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:16.330447 10501 hybrid_clock.cc:648] HybridClock initialized: now 1786050556330409 us; error 54 us; skew 500 ppm
Aug 06 21:09:16 dist-test-slave-69rg krb5kdc[10114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050556, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:16.333662 10501 init.cc:377] Logged in from keytab as kudu/127.27.225.195@KRBTEST.COM (short username kudu)
I20260806 21:09:16.334726 10501 webserver.cc:460] Webserver started at http://127.27.225.195:39733/ using document root <none> and password file <none>
I20260806 21:09:16.335328 10501 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:16.335424 10501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:16.335673 10501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:09:16.337795 10501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "44df30bca10d4c36a0d1c41f8bff84dd"
format_stamp: "Formatted at 2026-08-06 21:09:16 on dist-test-slave-69rg"
server_key: "3e56eb8ad2b5391f3e8800b67317299c"
server_key_iv: "daed8bcf27530384a3495a5fc8ad3de4"
server_key_version: "encryptionkey@0"
I20260806 21:09:16.338493 10501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "44df30bca10d4c36a0d1c41f8bff84dd"
format_stamp: "Formatted at 2026-08-06 21:09:16 on dist-test-slave-69rg"
server_key: "3e56eb8ad2b5391f3e8800b67317299c"
server_key_iv: "daed8bcf27530384a3495a5fc8ad3de4"
server_key_version: "encryptionkey@0"
I20260806 21:09:16.342293 10501 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:09:16.344735 10516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:16.346122 10501 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:09:16.346259 10501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "44df30bca10d4c36a0d1c41f8bff84dd"
format_stamp: "Formatted at 2026-08-06 21:09:16 on dist-test-slave-69rg"
server_key: "3e56eb8ad2b5391f3e8800b67317299c"
server_key_iv: "daed8bcf27530384a3495a5fc8ad3de4"
server_key_version: "encryptionkey@0"
I20260806 21:09:16.346377 10501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:16.361758 10501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:16.365515 10501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:16.365751 10501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:16.366427 10501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:09:16.367331 10501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:09:16.367405 10501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:16.367481 10501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:09:16.367532 10501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:16.377439 10501 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:38791
I20260806 21:09:16.377449 10633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:38791 every 8 connection(s)
I20260806 21:09:16.378432 10501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:09:16.384824 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 10501
I20260806 21:09:16.384999 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:09:16.385286 28551 external_mini_cluster.cc:1428] Setting key 147cc1a0f89f133514a22a9c593d03b6
Aug 06 21:09:16 dist-test-slave-69rg krb5kdc[10114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050556, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:09:16.393980 10199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:16.380412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:50467 (local address 127.27.225.254:40157)
0806 21:09:16.380565 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:16.380569 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:16.381392 (+ 823us) server_negotiation.cc:407] Connection header received
0806 21:09:16.382909 (+ 1517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:16.382913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:16.382973 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:16.383084 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:16.384273 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.384808 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.386338 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.386660 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.386802 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:16.389803 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:16.389807 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:16.389809 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:16.389892 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:16.391486 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:16.392228 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:16.392236 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:16.392238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:16.392362 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:16.392750 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:16.392755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:16.392759 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:16.392972 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:16.393090 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:16.393636 (+ 546us) server_negotiation.cc:299] Negotiation successful
0806 21:09:16.393781 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 21:09:16.394943 10636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:16.380754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.195:50467)
0806 21:09:16.381231 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:16.381277 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:16.382694 (+ 1417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:16.383261 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:16.383268 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:16.383658 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:16.384128 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:16.384141 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.384940 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.384945 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:16.386174 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:16.386186 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.386874 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.386878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:16.387081 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:16.387867 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:16.387884 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:16.389581 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:16.391741 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:16.391774 (+ 33us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:16.391789 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:16.392097 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:16.392467 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:16.392471 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:16.392473 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:16.392641 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:16.393089 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:16.393098 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:16.393381 (+ 283us) client_negotiation.cc:769] Sending connection context
0806 21:09:16.393591 (+ 210us) client_negotiation.cc:240] Negotiation successful
0806 21:09:16.393992 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":121,"threads_started":1}
I20260806 21:09:16.396415 10634 heartbeater.cc:347] Connected to a master server at 127.27.225.254:40157
I20260806 21:09:16.396646 10634 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:16.397173 10634 heartbeater.cc:511] Master 127.27.225.254:40157 requested a full tablet report, sending...
I20260806 21:09:16.398361 10160 ts_manager.cc:194] Registered new tserver with Master: 44df30bca10d4c36a0d1c41f8bff84dd (127.27.225.195:38791)
I20260806 21:09:16.399158 10160 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.195@KRBTEST.COM'} at 127.27.225.195:50467
I20260806 21:09:16.399751 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:09:16.409155 10199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:16.401923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51724 (local address 127.27.225.254:40157)
0806 21:09:16.402045 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:16.402055 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:16.402229 (+ 174us) server_negotiation.cc:407] Connection header received
0806 21:09:16.402323 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:16.402325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:16.402362 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:16.402430 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:16.403072 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.403592 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.404408 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.404614 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.404690 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:16.406157 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:16.406160 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:16.406162 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:16.406186 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:16.407682 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:16.408119 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:16.408121 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:16.408123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:16.408198 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:16.408441 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:16.408443 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:16.408444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:16.408660 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:16.408766 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:16.408939 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 21:09:16.409033 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 21:09:16.412457 10154 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51724:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:09:16.415627 10154 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:09:16.430521 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:16.424636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39272 (local address 127.27.225.195:38791)
0806 21:09:16.425255 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:16.425267 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:16.425642 (+ 375us) server_negotiation.cc:407] Connection header received
0806 21:09:16.425731 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:16.425736 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:16.425973 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:16.426126 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:09:16.426758 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.427484 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.429183 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.429753 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.429923 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:16.430072 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:16.430239 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 21:09:16.430360 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":120,"threads_started":1}
I20260806 21:09:16.430505 10649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:16.424750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:35613 (local address 127.0.0.1:42468)
0806 21:09:16.425391 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:16.425406 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:16.425517 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:16.426307 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:16.426311 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:16.426345 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:09:16.426580 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:16.426587 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.427610 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.427613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:16.428872 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:16.428884 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.430166 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.430171 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:16.430268 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:16.430288 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 21:09:16.430355 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 21:09:16.430408 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"mutex_wait_us":154,"spinlock_wait_cycles":13952,"thread_start_us":99,"threads_started":1}
I20260806 21:09:16.430505 10645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:16.424480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:38791 (local address 127.0.0.1:39272)
0806 21:09:16.425300 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:16.425357 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:16.425498 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:16.426294 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:16.426303 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:16.426341 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:09:16.426593 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:16.426600 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.427655 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.427658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:16.429038 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:16.429048 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.429921 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.429925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:16.430034 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:16.430071 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 21:09:16.430191 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 21:09:16.430323 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"mutex_wait_us":142,"spinlock_wait_cycles":2304,"thread_start_us":82,"threads_started":1}
I20260806 21:09:16.431200 10650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:16.425223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42468 (local address 127.27.225.193:35613)
0806 21:09:16.425787 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:16.425793 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:16.425833 (+ 40us) server_negotiation.cc:407] Connection header received
0806 21:09:16.425898 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:16.425903 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:16.426053 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:16.426168 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:09:16.426718 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.427475 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.429014 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.429864 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.429981 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:16.430095 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:16.430890 (+ 795us) server_negotiation.cc:299] Negotiation successful
0806 21:09:16.431017 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":115,"threads_started":1}
I20260806 21:09:16.432361 10647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:16.424480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:43135 (local address 127.0.0.1:46696)
0806 21:09:16.425420 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:16.425433 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:16.425515 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:16.426337 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:16.426339 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:16.426352 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:09:16.426594 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:16.426600 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.428973 (+ 2373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.428977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:16.430209 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:16.430220 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.431892 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.431898 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:16.432021 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:16.432049 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 21:09:16.432114 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 21:09:16.432198 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":868,"mutex_wait_us":669,"spinlock_wait_cycles":8320,"thread_start_us":110,"threads_started":1}
I20260806 21:09:16.433171 10648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:16.424725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46696 (local address 127.27.225.194:43135)
0806 21:09:16.425793 (+ 1068us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:16.425809 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:16.425869 (+ 60us) server_negotiation.cc:407] Connection header received
0806 21:09:16.425943 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:16.425948 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:16.426161 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:16.426362 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:09:16.427638 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.428819 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.430348 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.431258 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.432583 (+ 1325us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:16.432866 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:16.432926 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 21:09:16.433024 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":962,"thread_start_us":129,"threads_started":1}
I20260806 21:09:16.434873 10566 tablet_service.cc:1467] Processing CreateTablet for tablet ca6bcdd28aef48d69cc13bbf51bed3d3 (DEFAULT_TABLE table=test-table [id=b421f7debfd34065a76c7e7d33f19627]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:09:16.434789 10288 tablet_service.cc:1467] Processing CreateTablet for tablet ca6bcdd28aef48d69cc13bbf51bed3d3 (DEFAULT_TABLE table=test-table [id=b421f7debfd34065a76c7e7d33f19627]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:09:16.436005 10288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca6bcdd28aef48d69cc13bbf51bed3d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:09:16.436015 10566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca6bcdd28aef48d69cc13bbf51bed3d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:09:16.435873 10427 tablet_service.cc:1467] Processing CreateTablet for tablet ca6bcdd28aef48d69cc13bbf51bed3d3 (DEFAULT_TABLE table=test-table [id=b421f7debfd34065a76c7e7d33f19627]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:09:16.436884 10427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca6bcdd28aef48d69cc13bbf51bed3d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:09:16.442705 10651 tablet_bootstrap.cc:492] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d: Bootstrap starting.
I20260806 21:09:16.444201 10653 tablet_bootstrap.cc:492] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52: Bootstrap starting.
I20260806 21:09:16.445008 10651 tablet_bootstrap.cc:654] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d: Neither blocks nor log segments found. Creating new log.
I20260806 21:09:16.445749 10651 log.cc:824] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d: Log is configured to *not* fsync() on all Append() calls
I20260806 21:09:16.446204 10652 tablet_bootstrap.cc:492] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 44df30bca10d4c36a0d1c41f8bff84dd: Bootstrap starting.
I20260806 21:09:16.446520 10653 tablet_bootstrap.cc:654] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52: Neither blocks nor log segments found. Creating new log.
I20260806 21:09:16.447348 10653 log.cc:824] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52: Log is configured to *not* fsync() on all Append() calls
I20260806 21:09:16.448127 10651 tablet_bootstrap.cc:492] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d: No bootstrap required, opened a new log
I20260806 21:09:16.448331 10651 ts_tablet_manager.cc:1397] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260806 21:09:16.448625 10652 tablet_bootstrap.cc:654] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 44df30bca10d4c36a0d1c41f8bff84dd: Neither blocks nor log segments found. Creating new log.
I20260806 21:09:16.449357 10652 log.cc:824] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 44df30bca10d4c36a0d1c41f8bff84dd: Log is configured to *not* fsync() on all Append() calls
I20260806 21:09:16.449394 10653 tablet_bootstrap.cc:492] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52: No bootstrap required, opened a new log
I20260806 21:09:16.449590 10653 ts_tablet_manager.cc:1397] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:09:16.451445 10652 tablet_bootstrap.cc:492] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 44df30bca10d4c36a0d1c41f8bff84dd: No bootstrap required, opened a new log
I20260806 21:09:16.451634 10652 ts_tablet_manager.cc:1397] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 44df30bca10d4c36a0d1c41f8bff84dd: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 21:09:16.453172 10651 raft_consensus.cc:348] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:16.453469 10651 raft_consensus.cc:374] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:09:16.453526 10651 raft_consensus.cc:729] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04cf9ddb3b194f778e793f21766db37d, State: Initialized, Role: FOLLOWER
I20260806 21:09:16.453457 10653 raft_consensus.cc:348] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:16.453692 10653 raft_consensus.cc:374] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:09:16.453770 10653 raft_consensus.cc:729] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40237e8dc68b4fc6ada088cfba203a52, State: Initialized, Role: FOLLOWER
I20260806 21:09:16.454021 10651 consensus_queue.cc:260] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:16.454154 10653 consensus_queue.cc:260] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:16.454883 10652 raft_consensus.cc:348] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 44df30bca10d4c36a0d1c41f8bff84dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:16.455080 10651 ts_tablet_manager.cc:1428] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260806 21:09:16.455080 10653 ts_tablet_manager.cc:1428] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 21:09:16.455122 10652 raft_consensus.cc:374] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 44df30bca10d4c36a0d1c41f8bff84dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:09:16.455189 10652 raft_consensus.cc:729] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 44df30bca10d4c36a0d1c41f8bff84dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44df30bca10d4c36a0d1c41f8bff84dd, State: Initialized, Role: FOLLOWER
I20260806 21:09:16.455435 10496 heartbeater.cc:503] Master 127.27.225.254:40157 was elected leader, sending a full tablet report...
I20260806 21:09:16.455435 10356 heartbeater.cc:503] Master 127.27.225.254:40157 was elected leader, sending a full tablet report...
I20260806 21:09:16.455598 10652 consensus_queue.cc:260] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 44df30bca10d4c36a0d1c41f8bff84dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:16.456223 10634 heartbeater.cc:503] Master 127.27.225.254:40157 was elected leader, sending a full tablet report...
I20260806 21:09:16.457453 10652 ts_tablet_manager.cc:1428] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 44df30bca10d4c36a0d1c41f8bff84dd: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 21:09:16.519770 10658 raft_consensus.cc:482] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:09:16.520023 10658 raft_consensus.cc:504] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:16.521260 10658 leader_election.cc:288] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44df30bca10d4c36a0d1c41f8bff84dd (127.27.225.195:38791), 04cf9ddb3b194f778e793f21766db37d (127.27.225.193:35613)
I20260806 21:09:16.525564 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:16.521694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:57137 (local address 127.27.225.195:38791)
0806 21:09:16.521864 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:16.521869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:16.521887 (+ 18us) server_negotiation.cc:407] Connection header received
0806 21:09:16.521951 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:16.521956 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:16.522007 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:16.522099 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:09:16.522550 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.523269 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.524580 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.524999 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.525149 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:16.525206 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:16.525378 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 21:09:16.525450 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 21:09:16.525564 10498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:16.521604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:38791 (local address 127.27.225.194:57137)
0806 21:09:16.521716 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:16.521731 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:16.521854 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:16.522202 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:16.522208 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:16.522229 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:09:16.522406 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:16.522415 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.523391 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.523394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:16.524458 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:16.524467 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.525137 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.525141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:16.525247 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:16.525273 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 21:09:16.525360 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 21:09:16.525433 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 21:09:16.526793 10661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:16.521604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:35613 (local address 127.27.225.194:54313)
0806 21:09:16.522529 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:16.522543 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:16.522726 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:16.523020 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:16.523023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:16.523042 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:09:16.523152 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:16.523157 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.524191 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.524195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:16.525313 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:16.525324 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.526390 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.526393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:16.526479 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:16.526500 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 21:09:16.526580 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 21:09:16.526645 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":865,"mutex_wait_us":496,"thread_start_us":141,"threads_started":1}
I20260806 21:09:16.526809 10650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:16.521897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:54313 (local address 127.27.225.193:35613)
0806 21:09:16.522073 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:16.522079 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:16.522652 (+ 573us) server_negotiation.cc:407] Connection header received
0806 21:09:16.522839 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:16.522844 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:16.522923 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:16.523003 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:09:16.523259 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.524027 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.525455 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.526132 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.526227 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:16.526279 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:16.526640 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 21:09:16.526718 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 21:09:16.527390 10587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca6bcdd28aef48d69cc13bbf51bed3d3" candidate_uuid: "40237e8dc68b4fc6ada088cfba203a52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" is_pre_election: true
I20260806 21:09:16.527719 10587 raft_consensus.cc:2454] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 44df30bca10d4c36a0d1c41f8bff84dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40237e8dc68b4fc6ada088cfba203a52 in term 0.
I20260806 21:09:16.527721 10308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca6bcdd28aef48d69cc13bbf51bed3d3" candidate_uuid: "40237e8dc68b4fc6ada088cfba203a52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04cf9ddb3b194f778e793f21766db37d" is_pre_election: true
I20260806 21:09:16.527997 10308 raft_consensus.cc:2454] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40237e8dc68b4fc6ada088cfba203a52 in term 0.
I20260806 21:09:16.528242 10381 leader_election.cc:302] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40237e8dc68b4fc6ada088cfba203a52, 44df30bca10d4c36a0d1c41f8bff84dd; no voters:
I20260806 21:09:16.528514 10658 raft_consensus.cc:2781] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:09:16.528616 10658 raft_consensus.cc:482] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:09:16.528673 10658 raft_consensus.cc:3037] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:09:16.529734 10658 raft_consensus.cc:504] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:16.530149 10658 leader_election.cc:288] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52 [CANDIDATE]: Term 1 election: Requested vote from peers 44df30bca10d4c36a0d1c41f8bff84dd (127.27.225.195:38791), 04cf9ddb3b194f778e793f21766db37d (127.27.225.193:35613)
I20260806 21:09:16.530550 10587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca6bcdd28aef48d69cc13bbf51bed3d3" candidate_uuid: "40237e8dc68b4fc6ada088cfba203a52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44df30bca10d4c36a0d1c41f8bff84dd"
I20260806 21:09:16.530550 10308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca6bcdd28aef48d69cc13bbf51bed3d3" candidate_uuid: "40237e8dc68b4fc6ada088cfba203a52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04cf9ddb3b194f778e793f21766db37d"
I20260806 21:09:16.530686 10587 raft_consensus.cc:3037] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 44df30bca10d4c36a0d1c41f8bff84dd [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:09:16.530706 10308 raft_consensus.cc:3037] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:09:16.532007 10587 raft_consensus.cc:2454] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 44df30bca10d4c36a0d1c41f8bff84dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40237e8dc68b4fc6ada088cfba203a52 in term 1.
I20260806 21:09:16.532006 10308 raft_consensus.cc:2454] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40237e8dc68b4fc6ada088cfba203a52 in term 1.
I20260806 21:09:16.532338 10379 leader_election.cc:302] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04cf9ddb3b194f778e793f21766db37d, 40237e8dc68b4fc6ada088cfba203a52; no voters:
I20260806 21:09:16.532523 10658 raft_consensus.cc:2781] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:09:16.532783 10658 raft_consensus.cc:686] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52 [term 1 LEADER]: Becoming Leader. State: Replica: 40237e8dc68b4fc6ada088cfba203a52, State: Running, Role: LEADER
I20260806 21:09:16.533159 10658 consensus_queue.cc:237] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:16.536300 10160 catalog_manager.cc:5495] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52 reported cstate change: term changed from 0 to 1, leader changed from <none> to 40237e8dc68b4fc6ada088cfba203a52 (127.27.225.194). New cstate: current_term: 1 leader_uuid: "40237e8dc68b4fc6ada088cfba203a52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } health_report { overall_health: UNKNOWN } } }
I20260806 21:09:16.555382 10648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:16.551771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46712 (local address 127.27.225.194:43135)
0806 21:09:16.551908 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:16.551912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:16.551926 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:09:16.552041 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:16.552043 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:16.552104 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:16.552195 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:09:16.552416 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.552912 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.553915 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:16.554137 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:16.554270 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:16.554558 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:09:16.554997 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:09:16.555087 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:16.555214 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 21:09:16.555268 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 21:09:16.561522 10587 raft_consensus.cc:1261] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 44df30bca10d4c36a0d1c41f8bff84dd [term 1 FOLLOWER]: Refusing update from remote peer 40237e8dc68b4fc6ada088cfba203a52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:09:16.561522 10308 raft_consensus.cc:1261] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d [term 1 FOLLOWER]: Refusing update from remote peer 40237e8dc68b4fc6ada088cfba203a52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:09:16.562461 10658 consensus_queue.cc:1035] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:09:16.562719 10662 consensus_queue.cc:1035] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:09:16.571439 10666 mvcc.cc:205] Tried to move back new op lower bound from 7315663079667699712 to 7315663079562616832. Current Snapshot: MvccSnapshot[applied={T|T < 7315663079667699712}]
I20260806 21:09:16.574043 10667 mvcc.cc:205] Tried to move back new op lower bound from 7315663079667699712 to 7315663079562616832. Current Snapshot: MvccSnapshot[applied={T|T < 7315663079667699712}]
I20260806 21:09:16.580072 10159 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:09:16.580293 10159 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:09:16.582870 10159 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38: Sending DeleteTablet for 3 replicas of tablet ca6bcdd28aef48d69cc13bbf51bed3d3
I20260806 21:09:16.583897 10288 tablet_service.cc:1514] Processing DeleteTablet for tablet ca6bcdd28aef48d69cc13bbf51bed3d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:09:16 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:42468
I20260806 21:09:16.583943 10427 tablet_service.cc:1514] Processing DeleteTablet for tablet ca6bcdd28aef48d69cc13bbf51bed3d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:09:16 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:46696
I20260806 21:09:16.584033 10566 tablet_service.cc:1514] Processing DeleteTablet for tablet ca6bcdd28aef48d69cc13bbf51bed3d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:09:16 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:39272
I20260806 21:09:16.584512 10674 tablet_replica.cc:330] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52: stopping tablet replica
I20260806 21:09:16.584609 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 10223
I20260806 21:09:16.584883 10674 raft_consensus.cc:2229] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:09:16.585984 10677 tablet_replica.cc:330] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 44df30bca10d4c36a0d1c41f8bff84dd: stopping tablet replica
I20260806 21:09:16.586388 10674 raft_consensus.cc:2258] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:09:16.586395 10677 raft_consensus.cc:2229] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 44df30bca10d4c36a0d1c41f8bff84dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:09:16.586802 10677 raft_consensus.cc:2258] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 44df30bca10d4c36a0d1c41f8bff84dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:09:16.588722 10677 ts_tablet_manager.cc:1905] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 44df30bca10d4c36a0d1c41f8bff84dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:09:16.588900 10674 ts_tablet_manager.cc:1905] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:09:16.591982 10674 ts_tablet_manager.cc:1918] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:09:16.592092 10674 log.cc:1196] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/wal/wals/ca6bcdd28aef48d69cc13bbf51bed3d3
I20260806 21:09:16.592424 10674 ts_tablet_manager.cc:1939] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 40237e8dc68b4fc6ada088cfba203a52: Deleting consensus metadata
I20260806 21:09:16.592834 10677 ts_tablet_manager.cc:1918] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 44df30bca10d4c36a0d1c41f8bff84dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:09:16.593019 10677 log.cc:1196] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 44df30bca10d4c36a0d1c41f8bff84dd: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/ca6bcdd28aef48d69cc13bbf51bed3d3
I20260806 21:09:16.593462 10677 ts_tablet_manager.cc:1939] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 44df30bca10d4c36a0d1c41f8bff84dd: Deleting consensus metadata
I20260806 21:09:16.593787 10143 catalog_manager.cc:4841] TS 40237e8dc68b4fc6ada088cfba203a52 (127.27.225.194:43135): tablet ca6bcdd28aef48d69cc13bbf51bed3d3 (table test-table [id=b421f7debfd34065a76c7e7d33f19627]) successfully deleted
W20260806 21:09:16.594386 10144 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.225.193:35613 (error 108)
I20260806 21:09:16.594681 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 10361
I20260806 21:09:16.595667 10147 catalog_manager.cc:4841] TS 44df30bca10d4c36a0d1c41f8bff84dd (127.27.225.195:38791): tablet ca6bcdd28aef48d69cc13bbf51bed3d3 (table test-table [id=b421f7debfd34065a76c7e7d33f19627]) successfully deleted
I20260806 21:09:16.596880 10647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:16.596507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:35613 (local address 127.0.0.1:42474)
0806 21:09:16.596704 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:16.596779 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.193:35613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260806 21:09:16.597132 10144 catalog_manager.cc:4584] TS 04cf9ddb3b194f778e793f21766db37d (127.27.225.193:35613): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ca6bcdd28aef48d69cc13bbf51bed3d3: Network error: Client connection negotiation failed: client connection to 127.27.225.193:35613: connect: Connection refused (error 111)
I20260806 21:09:16.602860 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 10501
I20260806 21:09:16.609501 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 10127
I20260806 21:09:16.617756 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:40157
--webserver_interface=127.27.225.254
--webserver_port=37955
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:36065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:40157
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:09:16.741766 10679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:16.742122 10679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:16.742167 10679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:16.746551 10679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:09:16.746635 10679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:16.746655 10679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:16.746675 10679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:09:16.746691 10679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:09:16.752424 10679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:36065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:40157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:40157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=37955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:16.753612 10679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:16.754739 10679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:16.769863 10686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:16.769914 10679 server_base.cc:1034] running on GCE node
W20260806 21:09:16.769865 10685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:16.769866 10688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:16.770780 10679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:16.772102 10679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:16.773320 10679 hybrid_clock.cc:648] HybridClock initialized: now 1786050556773304 us; error 30 us; skew 500 ppm
Aug 06 21:09:16 dist-test-slave-69rg krb5kdc[10114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050556, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:16.776506 10679 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:09:16.777621 10679 webserver.cc:460] Webserver started at http://127.27.225.254:37955/ using document root <none> and password file <none>
I20260806 21:09:16.778332 10679 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:16.778458 10679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:16.782793 10679 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 21:09:16.785354 10696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:16.786733 10679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:09:16.786918 10679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "a8326cac2b564a289b3a80bca33e7d38"
format_stamp: "Formatted at 2026-08-06 21:09:15 on dist-test-slave-69rg"
server_key: "1f1319a8ecda1ae6596ac9cad82df334"
server_key_iv: "d33404251e8182f753e8bb88219dcb48"
server_key_version: "encryptionkey@0"
I20260806 21:09:16.787633 10679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:16.812503 10679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:16.816138 10679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:16.816403 10679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:16.824581 10679 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:40157
I20260806 21:09:16.825021 10750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:40157 every 8 connection(s)
I20260806 21:09:16.825840 10679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:09:16.826274 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 10679
I20260806 21:09:16.826756 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:35613
--local_ip_for_outbound_sockets=127.27.225.193
--tserver_master_addrs=127.27.225.254:40157
--webserver_port=35679
--webserver_interface=127.27.225.193
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:36065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:09:16.830413 10751 sys_catalog.cc:263] Verifying existing consensus state
I20260806 21:09:16.832068 10751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38: Bootstrap starting.
I20260806 21:09:16.850159 10751 log.cc:824] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38: Log is configured to *not* fsync() on all Append() calls
I20260806 21:09:16.859362 10751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 21:09:16.859833 10751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38: Bootstrap complete.
I20260806 21:09:16.864286 10751 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8326cac2b564a289b3a80bca33e7d38" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40157 } }
I20260806 21:09:16.864722 10751 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8326cac2b564a289b3a80bca33e7d38, State: Initialized, Role: FOLLOWER
I20260806 21:09:16.865285 10751 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8326cac2b564a289b3a80bca33e7d38" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40157 } }
I20260806 21:09:16.865410 10751 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:09:16.865507 10751 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:09:16.865625 10751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [term 1 FOLLOWER]: Advancing to term 2
I20260806 21:09:16.868505 10751 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8326cac2b564a289b3a80bca33e7d38" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40157 } }
I20260806 21:09:16.868913 10751 leader_election.cc:302] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8326cac2b564a289b3a80bca33e7d38; no voters:
I20260806 21:09:16.869243 10751 leader_election.cc:288] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 21:09:16.869339 10755 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 21:09:16.869561 10755 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [term 2 LEADER]: Becoming Leader. State: Replica: a8326cac2b564a289b3a80bca33e7d38, State: Running, Role: LEADER
I20260806 21:09:16.869939 10755 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8326cac2b564a289b3a80bca33e7d38" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40157 } }
I20260806 21:09:16.870640 10751 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:09:16.870954 10756 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a8326cac2b564a289b3a80bca33e7d38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8326cac2b564a289b3a80bca33e7d38" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40157 } } }
I20260806 21:09:16.870967 10755 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8326cac2b564a289b3a80bca33e7d38. Latest consensus state: current_term: 2 leader_uuid: "a8326cac2b564a289b3a80bca33e7d38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8326cac2b564a289b3a80bca33e7d38" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40157 } } }
I20260806 21:09:16.871223 10756 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [sys.catalog]: This master's current role is: LEADER
I20260806 21:09:16.871258 10755 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [sys.catalog]: This master's current role is: LEADER
I20260806 21:09:16.871709 10760 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:09:16.877118 10760 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:09:16.877933 10760 catalog_manager.cc:1259] Loaded cluster ID: f62d58e3dd3441cd86ccc5ffed368f7a
I20260806 21:09:16.877995 10760 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:09:16.879420 10760 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 21:09:16.883945 10760 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 21:09:16.889878 10760 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:09:16.890764 10760 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38: Loaded TSK: 0
I20260806 21:09:16.892230 10760 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 21:09:16.952646 10753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:16.952948 10753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:16.952991 10753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:16.957255 10753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:16.957334 10753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:16.957396 10753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:09:16.962983 10753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:36065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:35613
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=35679
--tserver_master_addrs=127.27.225.254:40157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:16.964056 10753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:16.965070 10753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:16.976413 10779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:16.976435 10778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:16.976679 10781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:16.976936 10753 server_base.cc:1034] running on GCE node
I20260806 21:09:16.977442 10753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:16.978271 10753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:16.979488 10753 hybrid_clock.cc:648] HybridClock initialized: now 1786050556979464 us; error 45 us; skew 500 ppm
Aug 06 21:09:16 dist-test-slave-69rg krb5kdc[10114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050556, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:16.982574 10753 init.cc:377] Logged in from keytab as kudu/127.27.225.193@KRBTEST.COM (short username kudu)
I20260806 21:09:16.983516 10753 webserver.cc:460] Webserver started at http://127.27.225.193:35679/ using document root <none> and password file <none>
I20260806 21:09:16.984149 10753 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:16.984226 10753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:16.988559 10753 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 21:09:16.991207 10788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:16.992852 10753 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:09:16.993024 10753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "04cf9ddb3b194f778e793f21766db37d"
format_stamp: "Formatted at 2026-08-06 21:09:15 on dist-test-slave-69rg"
server_key: "424bbe2cb4e5e30ee84733d739b134a5"
server_key_iv: "b73fec191d17567b21271906298cc16a"
server_key_version: "encryptionkey@0"
I20260806 21:09:16.993630 10753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:17.007251 10753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:17.011587 10753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:17.011858 10753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:17.012604 10753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:09:17.013952 10795 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 21:09:17.018934 10753 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 21:09:17.019021 10753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260806 21:09:17.019078 10753 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 21:09:17.020974 10753 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 21:09:17.021045 10753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 21:09:17.021315 10795 tablet_bootstrap.cc:492] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d: Bootstrap starting.
I20260806 21:09:17.032763 10753 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:35613
I20260806 21:09:17.032792 10905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:35613 every 8 connection(s)
I20260806 21:09:17.034122 10753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:09:17.034842 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 10753
I20260806 21:09:17.035377 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:43135
--local_ip_for_outbound_sockets=127.27.225.194
--tserver_master_addrs=127.27.225.254:40157
--webserver_port=38499
--webserver_interface=127.27.225.194
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:36065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:09:17.044270 10795 log.cc:824] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d: Log is configured to *not* fsync() on all Append() calls
Aug 06 21:09:17 dist-test-slave-69rg krb5kdc[10114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050556, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:09:17.053248 10795 tablet_bootstrap.cc:492] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 21:09:17.053715 10795 tablet_bootstrap.cc:492] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d: Bootstrap complete.
I20260806 21:09:17.053890 10909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:17.036613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.193:42361)
0806 21:09:17.037553 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:17.037618 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:17.038935 (+ 1317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:17.040233 (+ 1298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:17.040241 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:17.040688 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:17.041134 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:17.041148 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.042579 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.042582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:17.043440 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:17.043451 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.044211 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.044216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:17.044416 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:17.045149 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:17.045188 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:17.047347 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:17.050211 (+ 2864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:17.050217 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:17.050231 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:17.050605 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:17.051041 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:17.051044 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:17.051046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:17.051225 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:17.051949 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:17.051957 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:17.052256 (+ 299us) client_negotiation.cc:769] Sending connection context
0806 21:09:17.052454 (+ 198us) client_negotiation.cc:240] Negotiation successful
0806 21:09:17.052785 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"thread_start_us":181,"threads_started":1}
I20260806 21:09:17.053903 10908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:17.036638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:42361 (local address 127.27.225.254:40157)
0806 21:09:17.037183 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:17.037208 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:17.039261 (+ 2053us) server_negotiation.cc:407] Connection header received
0806 21:09:17.039513 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:17.039522 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:17.039833 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:17.040174 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:17.041335 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.042407 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.043581 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.044060 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.044244 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:17.047609 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:17.047618 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:17.047637 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:17.047672 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:17.050043 (+ 2371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:17.050746 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:17.050752 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:17.050758 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:17.050931 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:17.051336 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:17.051339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:17.051341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:17.051816 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:17.051953 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:17.052612 (+ 659us) server_negotiation.cc:299] Negotiation successful
0806 21:09:17.052920 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":127,"threads_started":1}
I20260806 21:09:17.055042 10795 ts_tablet_manager.cc:1397] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d: Time spent bootstrapping tablet: real 0.034s user 0.020s sys 0.012s
I20260806 21:09:17.056630 10906 heartbeater.cc:347] Connected to a master server at 127.27.225.254:40157
I20260806 21:09:17.056846 10906 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:17.057387 10906 heartbeater.cc:511] Master 127.27.225.254:40157 requested a full tablet report, sending...
I20260806 21:09:17.058043 10795 raft_consensus.cc:348] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:17.058579 10795 raft_consensus.cc:729] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04cf9ddb3b194f778e793f21766db37d, State: Initialized, Role: FOLLOWER
I20260806 21:09:17.059095 10795 consensus_queue.cc:260] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:17.060133 10795 ts_tablet_manager.cc:1428] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 21:09:17.060127 10715 ts_manager.cc:194] Registered new tserver with Master: 04cf9ddb3b194f778e793f21766db37d (127.27.225.193:35613)
I20260806 21:09:17.063819 10715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.193@KRBTEST.COM'} at 127.27.225.193:42361
I20260806 21:09:17.066540 10906 heartbeater.cc:503] Master 127.27.225.254:40157 was elected leader, sending a full tablet report...
Aug 06 21:09:17 dist-test-slave-69rg krb5kdc[10114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050556, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for kudu/127.27.225.193@KRBTEST.COM
I20260806 21:09:17.074623 10913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:17.063178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:35613 (local address 127.0.0.1:42490)
0806 21:09:17.063784 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:17.063837 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:17.064077 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:17.064644 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:17.064649 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:17.065084 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:17.065231 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:17.065239 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.066609 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.066612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:17.067573 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:17.067586 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.068486 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.068489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:17.068571 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:17.069162 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:17.069174 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:17.070640 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:17.072816 (+ 2176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:17.072825 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:17.072831 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:17.073121 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:17.073444 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:17.073446 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:17.073448 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:17.073529 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:17.074086 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:17.074091 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:17.074199 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 21:09:17.074318 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 21:09:17.074481 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":131,"threads_started":1}
I20260806 21:09:17.074769 10915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:17.063268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42490 (local address 127.27.225.193:35613)
0806 21:09:17.063988 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:17.064000 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:17.064026 (+ 26us) server_negotiation.cc:407] Connection header received
0806 21:09:17.064272 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:17.064277 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:17.064478 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:17.064625 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:17.065374 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.066482 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.067717 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.068245 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.068353 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:17.070811 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:17.070822 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:17.070835 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:17.070862 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:17.072694 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:17.073240 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:17.073247 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:17.073248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:17.073348 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:17.073617 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:17.073620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:17.073622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:17.073971 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:17.074102 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:17.074421 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 21:09:17.074594 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":619,"thread_start_us":108,"threads_started":1}
I20260806 21:09:17.075779 10838 tablet_service.cc:1514] Processing DeleteTablet for tablet ca6bcdd28aef48d69cc13bbf51bed3d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:09:16 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:42490
I20260806 21:09:17.076736 10916 tablet_replica.cc:330] stopping tablet replica
I20260806 21:09:17.077106 10916 raft_consensus.cc:2229] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:09:17.077395 10916 raft_consensus.cc:2258] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:09:17.078608 10916 ts_tablet_manager.cc:1905] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:09:17.083434 10916 ts_tablet_manager.cc:1918] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:09:17.083551 10916 log.cc:1196] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/wal/wals/ca6bcdd28aef48d69cc13bbf51bed3d3
I20260806 21:09:17.083837 10916 ts_tablet_manager.cc:1939] T ca6bcdd28aef48d69cc13bbf51bed3d3 P 04cf9ddb3b194f778e793f21766db37d: Deleting consensus metadata
I20260806 21:09:17.086045 10698 catalog_manager.cc:4841] TS 04cf9ddb3b194f778e793f21766db37d (127.27.225.193:35613): tablet ca6bcdd28aef48d69cc13bbf51bed3d3 (table test-table [id=b421f7debfd34065a76c7e7d33f19627]) successfully deleted
W20260806 21:09:17.166425 10910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:17.166880 10910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:17.166978 10910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:17.171654 10910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:17.171749 10910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:17.171846 10910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:09:17.177697 10910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:36065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:43135
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=38499
--tserver_master_addrs=127.27.225.254:40157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:17.178978 10910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:17.180087 10910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:17.190835 10922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:17.190835 10924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:17.190836 10921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:17.191112 10910 server_base.cc:1034] running on GCE node
I20260806 21:09:17.191735 10910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:17.192538 10910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:17.193706 10910 hybrid_clock.cc:648] HybridClock initialized: now 1786050557193695 us; error 27 us; skew 500 ppm
Aug 06 21:09:17 dist-test-slave-69rg krb5kdc[10114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050557, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:17.196735 10910 init.cc:377] Logged in from keytab as kudu/127.27.225.194@KRBTEST.COM (short username kudu)
I20260806 21:09:17.197806 10910 webserver.cc:460] Webserver started at http://127.27.225.194:38499/ using document root <none> and password file <none>
I20260806 21:09:17.198514 10910 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:17.198611 10910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:17.202690 10910 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260806 21:09:17.204805 10931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:17.206212 10910 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:09:17.206363 10910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "40237e8dc68b4fc6ada088cfba203a52"
format_stamp: "Formatted at 2026-08-06 21:09:16 on dist-test-slave-69rg"
server_key: "ea5909bb6f9227fdd8115e213b96bb99"
server_key_iv: "c06b568a59b8c2d4c779ef16b9d924fe"
server_key_version: "encryptionkey@0"
I20260806 21:09:17.206928 10910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:17.219022 10910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:17.222678 10910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:17.222918 10910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:17.223560 10910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:09:17.224460 10910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:09:17.224535 10910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:17.224608 10910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:09:17.224661 10910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:17.234475 10910 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:43135
I20260806 21:09:17.234498 11047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:43135 every 8 connection(s)
I20260806 21:09:17.235530 10910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:09:17.245321 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 10910
Aug 06 21:09:17 dist-test-slave-69rg krb5kdc[10114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050557, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:09:17.245800 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:38791
--local_ip_for_outbound_sockets=127.27.225.195
--tserver_master_addrs=127.27.225.254:40157
--webserver_port=39733
--webserver_interface=127.27.225.195
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:36065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:09:17.250916 10908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:17.237623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:55743 (local address 127.27.225.254:40157)
0806 21:09:17.237788 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:17.237792 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:17.238632 (+ 840us) server_negotiation.cc:407] Connection header received
0806 21:09:17.240025 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:17.240028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:17.240091 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:17.240203 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:17.241475 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.242099 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.243262 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.243498 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.243636 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:17.246465 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:17.246468 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:17.246470 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:17.246497 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:17.248086 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:17.248845 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:17.248848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:17.248849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:17.248940 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:17.249354 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:17.249360 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:17.249365 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:17.249659 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:17.249778 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:17.250460 (+ 682us) server_negotiation.cc:299] Negotiation successful
0806 21:09:17.250651 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 21:09:17.251689 11050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:17.237958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.194:55743)
0806 21:09:17.238453 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:17.238495 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:17.239800 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:17.240385 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:17.240391 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:17.240786 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:17.241311 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:17.241326 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.242232 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.242242 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:17.243128 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:17.243138 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.243696 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.243701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:17.243871 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:17.244488 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:17.244504 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:17.246265 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:17.248223 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:17.248232 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:17.248252 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:17.248681 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:17.249043 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:17.249046 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:17.249047 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:17.249227 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:17.249804 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:17.249810 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:17.250169 (+ 359us) client_negotiation.cc:769] Sending connection context
0806 21:09:17.250366 (+ 197us) client_negotiation.cc:240] Negotiation successful
0806 21:09:17.250677 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":154,"threads_started":1}
I20260806 21:09:17.253063 11048 heartbeater.cc:347] Connected to a master server at 127.27.225.254:40157
I20260806 21:09:17.253319 11048 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:17.253937 11048 heartbeater.cc:511] Master 127.27.225.254:40157 requested a full tablet report, sending...
I20260806 21:09:17.255057 10715 ts_manager.cc:194] Registered new tserver with Master: 40237e8dc68b4fc6ada088cfba203a52 (127.27.225.194:43135)
I20260806 21:09:17.255697 10715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.194@KRBTEST.COM'} at 127.27.225.194:55743
W20260806 21:09:17.369988 11051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:17.370267 11051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:17.370352 11051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:17.374986 11051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:17.375058 11051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:17.375154 11051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:09:17.381105 11051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:36065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:38791
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=39733
--tserver_master_addrs=127.27.225.254:40157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:17.382357 11051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:17.383426 11051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:17.394254 11059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:17.394254 11056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:17.394264 11057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:17.394637 11051 server_base.cc:1034] running on GCE node
I20260806 21:09:17.395116 11051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:17.395862 11051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:17.397091 11051 hybrid_clock.cc:648] HybridClock initialized: now 1786050557397055 us; error 54 us; skew 500 ppm
Aug 06 21:09:17 dist-test-slave-69rg krb5kdc[10114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050557, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:17.400835 11051 init.cc:377] Logged in from keytab as kudu/127.27.225.195@KRBTEST.COM (short username kudu)
I20260806 21:09:17.402122 11051 webserver.cc:460] Webserver started at http://127.27.225.195:39733/ using document root <none> and password file <none>
I20260806 21:09:17.402736 11051 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:17.402855 11051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:17.407786 11051 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 21:09:17.410290 11067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:17.411679 11051 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 21:09:17.411863 11051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "44df30bca10d4c36a0d1c41f8bff84dd"
format_stamp: "Formatted at 2026-08-06 21:09:16 on dist-test-slave-69rg"
server_key: "3e56eb8ad2b5391f3e8800b67317299c"
server_key_iv: "daed8bcf27530384a3495a5fc8ad3de4"
server_key_version: "encryptionkey@0"
I20260806 21:09:17.412477 11051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:17.432770 11051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:17.436300 11051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:17.436533 11051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:17.437184 11051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:09:17.438208 11051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:09:17.438282 11051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:17.438355 11051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:09:17.438400 11051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:17.448629 11051 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:38791
I20260806 21:09:17.448702 11183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:38791 every 8 connection(s)
I20260806 21:09:17.449843 11051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:09:17.455417 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 11051
Aug 06 21:09:17 dist-test-slave-69rg krb5kdc[10114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050557, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:09:17.464967 11188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:17.456237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51740 (local address 127.27.225.254:40157)
0806 21:09:17.456534 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:17.456538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:17.456624 (+ 86us) server_negotiation.cc:407] Connection header received
0806 21:09:17.456780 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:17.456782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:17.456822 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:17.456899 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:17.457700 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.458490 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.459459 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.459712 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.459825 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:17.461496 (+ 1671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:17.461503 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:17.461509 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:17.461542 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:17.463029 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:17.463540 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:17.463542 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:17.463544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:17.463622 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:17.463930 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:17.463935 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:17.463936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:17.464174 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:17.464423 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:17.464599 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 21:09:17.464735 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":120,"threads_started":1}
I20260806 21:09:17.466147 10908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:17.451960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:60997 (local address 127.27.225.254:40157)
0806 21:09:17.452108 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:17.452117 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:17.452983 (+ 866us) server_negotiation.cc:407] Connection header received
0806 21:09:17.454498 (+ 1515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:17.454502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:17.454567 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:17.454647 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:17.456043 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.456825 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.458389 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.458684 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.458794 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:17.461871 (+ 3077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:17.461876 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:17.461879 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:17.461925 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:17.463692 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:17.464343 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:17.464346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:17.464348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:17.464437 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:17.464876 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:17.464880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:17.464882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:17.465086 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:17.465184 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:17.465873 (+ 689us) server_negotiation.cc:299] Negotiation successful
0806 21:09:17.465994 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 21:09:17.467126 11186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:17.452310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.195:60997)
0806 21:09:17.452819 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:17.452867 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:17.454264 (+ 1397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:17.454856 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:17.454863 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:17.455248 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:17.455827 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:17.455866 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.457008 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.457011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:17.458236 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:17.458258 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.458852 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.458855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:17.459014 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:17.459692 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:17.459709 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:17.461629 (+ 1920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:17.463831 (+ 2202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:17.463836 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:17.463851 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:17.464244 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:17.464552 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:17.464555 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:17.464557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:17.464746 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:17.465185 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:17.465193 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:17.465585 (+ 392us) client_negotiation.cc:769] Sending connection context
0806 21:09:17.465796 (+ 211us) client_negotiation.cc:240] Negotiation successful
0806 21:09:17.466181 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":140,"threads_started":1}
I20260806 21:09:17.468345 11184 heartbeater.cc:347] Connected to a master server at 127.27.225.254:40157
I20260806 21:09:17.468618 11184 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:17.469135 11184 heartbeater.cc:511] Master 127.27.225.254:40157 requested a full tablet report, sending...
I20260806 21:09:17.470243 10715 ts_manager.cc:194] Registered new tserver with Master: 44df30bca10d4c36a0d1c41f8bff84dd (127.27.225.195:38791)
I20260806 21:09:17.470841 10715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.195@KRBTEST.COM'} at 127.27.225.195:60997
I20260806 21:09:17.479290 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:09:17.483997 10908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:17.479936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51744 (local address 127.27.225.254:40157)
0806 21:09:17.480108 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:17.480113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:17.480286 (+ 173us) server_negotiation.cc:407] Connection header received
0806 21:09:17.480495 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:17.480498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:17.480611 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:17.480734 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:09:17.481003 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.481523 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.482514 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.482737 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.482851 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:17.483189 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:09:17.483590 (+ 401us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:09:17.483728 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:17.483792 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 21:09:17.483849 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 21:09:17.484669 10715 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:51744:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:09:17.488194 10715 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:09:17.496553 10838 tablet_service.cc:1467] Processing CreateTablet for tablet 586b330667c04e128907ff1fada91fd6 (DEFAULT_TABLE table=test-table [id=af7ead9dadd64c1d9afd87ea6892a439]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:09:17.497227 10838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 586b330667c04e128907ff1fada91fd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:09:17.500159 10795 tablet_bootstrap.cc:492] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d: Bootstrap starting.
I20260806 21:09:17.501113 10795 tablet_bootstrap.cc:654] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d: Neither blocks nor log segments found. Creating new log.
I20260806 21:09:17.501754 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:17.495845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:38791 (local address 127.0.0.1:39284)
0806 21:09:17.496069 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:17.496083 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:17.496167 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:17.497148 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:17.497151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:17.497170 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:09:17.497299 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:17.497307 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.499022 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.499026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:17.500154 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:17.500165 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.501264 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.501268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:17.501377 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:17.501425 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 21:09:17.501484 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 21:09:17.501562 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":40,"thread_start_us":97,"threads_started":1}
I20260806 21:09:17.501974 10913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:17.495441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:43135 (local address 127.0.0.1:46726)
0806 21:09:17.495568 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:17.495584 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:17.495663 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:17.496698 (+ 1035us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:17.496701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:17.496720 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:09:17.496845 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:17.496852 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.498226 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.498230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:17.499434 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:17.499446 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.501265 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.501268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:17.501591 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:17.501628 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 21:09:17.501722 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 21:09:17.501836 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":192}
I20260806 21:09:17.502059 11193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:17.495600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46726 (local address 127.27.225.194:43135)
0806 21:09:17.496106 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:17.496120 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:17.496143 (+ 23us) server_negotiation.cc:407] Connection header received
0806 21:09:17.496228 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:17.496234 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:17.496469 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:17.496776 (+ 307us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:09:17.496937 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.498095 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.499579 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.501111 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.501331 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:17.501554 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:17.501774 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 21:09:17.501926 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":158,"threads_started":1}
I20260806 21:09:17.502945 10795 tablet_bootstrap.cc:492] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d: No bootstrap required, opened a new log
I20260806 21:09:17.503058 10795 ts_tablet_manager.cc:1397] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:09:17.503291 11195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:17.496139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39284 (local address 127.27.225.195:38791)
0806 21:09:17.496526 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:17.496538 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:17.496561 (+ 23us) server_negotiation.cc:407] Connection header received
0806 21:09:17.496637 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:17.496641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:17.496829 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:17.497469 (+ 640us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:09:17.497680 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.498855 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.500331 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.501112 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.501981 (+ 869us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:17.502115 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:17.503018 (+ 903us) server_negotiation.cc:299] Negotiation successful
0806 21:09:17.503137 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":170,"threads_started":1}
I20260806 21:09:17.503589 10795 raft_consensus.cc:348] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:17.503724 10795 raft_consensus.cc:374] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:09:17.503770 10795 raft_consensus.cc:729] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04cf9ddb3b194f778e793f21766db37d, State: Initialized, Role: FOLLOWER
I20260806 21:09:17.503901 10795 consensus_queue.cc:260] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:17.504184 10795 ts_tablet_manager.cc:1428] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:09:17.505533 10980 tablet_service.cc:1467] Processing CreateTablet for tablet 586b330667c04e128907ff1fada91fd6 (DEFAULT_TABLE table=test-table [id=af7ead9dadd64c1d9afd87ea6892a439]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:09:17.506721 10980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 586b330667c04e128907ff1fada91fd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:09:17.514154 11197 tablet_bootstrap.cc:492] T 586b330667c04e128907ff1fada91fd6 P 40237e8dc68b4fc6ada088cfba203a52: Bootstrap starting.
I20260806 21:09:17.515885 11116 tablet_service.cc:1467] Processing CreateTablet for tablet 586b330667c04e128907ff1fada91fd6 (DEFAULT_TABLE table=test-table [id=af7ead9dadd64c1d9afd87ea6892a439]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:09:17.517083 11197 tablet_bootstrap.cc:654] T 586b330667c04e128907ff1fada91fd6 P 40237e8dc68b4fc6ada088cfba203a52: Neither blocks nor log segments found. Creating new log.
I20260806 21:09:17.517186 11116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 586b330667c04e128907ff1fada91fd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:09:17.518035 11197 log.cc:824] T 586b330667c04e128907ff1fada91fd6 P 40237e8dc68b4fc6ada088cfba203a52: Log is configured to *not* fsync() on all Append() calls
I20260806 21:09:17.520351 11197 tablet_bootstrap.cc:492] T 586b330667c04e128907ff1fada91fd6 P 40237e8dc68b4fc6ada088cfba203a52: No bootstrap required, opened a new log
I20260806 21:09:17.520565 11197 ts_tablet_manager.cc:1397] T 586b330667c04e128907ff1fada91fd6 P 40237e8dc68b4fc6ada088cfba203a52: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 21:09:17.524660 11197 raft_consensus.cc:348] T 586b330667c04e128907ff1fada91fd6 P 40237e8dc68b4fc6ada088cfba203a52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:17.524972 11197 raft_consensus.cc:374] T 586b330667c04e128907ff1fada91fd6 P 40237e8dc68b4fc6ada088cfba203a52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:09:17.525032 11197 raft_consensus.cc:729] T 586b330667c04e128907ff1fada91fd6 P 40237e8dc68b4fc6ada088cfba203a52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40237e8dc68b4fc6ada088cfba203a52, State: Initialized, Role: FOLLOWER
I20260806 21:09:17.525498 11197 consensus_queue.cc:260] T 586b330667c04e128907ff1fada91fd6 P 40237e8dc68b4fc6ada088cfba203a52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:17.526295 11048 heartbeater.cc:503] Master 127.27.225.254:40157 was elected leader, sending a full tablet report...
I20260806 21:09:17.526582 11197 ts_tablet_manager.cc:1428] T 586b330667c04e128907ff1fada91fd6 P 40237e8dc68b4fc6ada088cfba203a52: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 21:09:17.526702 11199 tablet_bootstrap.cc:492] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd: Bootstrap starting.
I20260806 21:09:17.529011 11199 tablet_bootstrap.cc:654] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd: Neither blocks nor log segments found. Creating new log.
I20260806 21:09:17.529774 11199 log.cc:824] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd: Log is configured to *not* fsync() on all Append() calls
I20260806 21:09:17.531795 11199 tablet_bootstrap.cc:492] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd: No bootstrap required, opened a new log
I20260806 21:09:17.532015 11199 ts_tablet_manager.cc:1397] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:09:17.535032 11199 raft_consensus.cc:348] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:17.535305 11199 raft_consensus.cc:374] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:09:17.535400 11199 raft_consensus.cc:729] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44df30bca10d4c36a0d1c41f8bff84dd, State: Initialized, Role: FOLLOWER
W20260806 21:09:17.535548 10907 tablet.cc:2369] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:09:17.535917 11199 consensus_queue.cc:260] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:17.536638 11184 heartbeater.cc:503] Master 127.27.225.254:40157 was elected leader, sending a full tablet report...
I20260806 21:09:17.536964 11199 ts_tablet_manager.cc:1428] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 21:09:17.678663 11202 raft_consensus.cc:482] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:09:17.678977 11202 raft_consensus.cc:504] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:17.680307 11202 leader_election.cc:288] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40237e8dc68b4fc6ada088cfba203a52 (127.27.225.194:43135), 04cf9ddb3b194f778e793f21766db37d (127.27.225.193:35613)
I20260806 21:09:17.684099 11193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:17.680655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:48669 (local address 127.27.225.194:43135)
0806 21:09:17.680834 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:17.680840 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:17.680863 (+ 23us) server_negotiation.cc:407] Connection header received
0806 21:09:17.681038 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:17.681041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:17.681086 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:17.681229 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:09:17.681511 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.682046 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.683109 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.683518 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.683653 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:17.683717 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:17.683904 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 21:09:17.683974 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 21:09:17.684099 11186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:17.680547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:43135 (local address 127.27.225.195:48669)
0806 21:09:17.680760 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:17.680776 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:17.680877 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:17.681238 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:17.681241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:17.681259 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:09:17.681379 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:17.681385 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.682168 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.682171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:17.682969 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:17.682978 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.683648 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.683652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:17.683775 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:17.683792 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 21:09:17.683885 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 21:09:17.683961 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132}
I20260806 21:09:17.684451 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:17.680781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:42025 (local address 127.27.225.193:35613)
0806 21:09:17.681122 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:17.681129 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:17.681145 (+ 16us) server_negotiation.cc:407] Connection header received
0806 21:09:17.681327 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:17.681331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:17.681392 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:17.681511 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:09:17.681795 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.682356 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.683396 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.683854 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.684000 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:17.684169 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:17.684214 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 21:09:17.684321 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":114,"threads_started":1}
I20260806 21:09:17.684690 11203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:17.680650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:35613 (local address 127.27.225.195:42025)
0806 21:09:17.681065 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:17.681079 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:17.681181 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:17.681522 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:17.681525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:17.681541 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:09:17.681663 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:17.681668 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.682479 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.682483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:17.683259 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:17.683269 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.683987 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.683991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:17.684080 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:17.684100 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 21:09:17.684469 (+ 369us) client_negotiation.cc:240] Negotiation successful
0806 21:09:17.684551 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":1,"thread_start_us":160,"threads_started":1}
I20260806 21:09:17.684894 11001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "586b330667c04e128907ff1fada91fd6" candidate_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40237e8dc68b4fc6ada088cfba203a52" is_pre_election: true
I20260806 21:09:17.685264 11001 raft_consensus.cc:2454] T 586b330667c04e128907ff1fada91fd6 P 40237e8dc68b4fc6ada088cfba203a52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44df30bca10d4c36a0d1c41f8bff84dd in term 0.
I20260806 21:09:17.685289 10859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "586b330667c04e128907ff1fada91fd6" candidate_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04cf9ddb3b194f778e793f21766db37d" is_pre_election: true
I20260806 21:09:17.685560 10859 raft_consensus.cc:2454] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44df30bca10d4c36a0d1c41f8bff84dd in term 0.
I20260806 21:09:17.685868 11068 leader_election.cc:302] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40237e8dc68b4fc6ada088cfba203a52, 44df30bca10d4c36a0d1c41f8bff84dd; no voters:
I20260806 21:09:17.686214 11202 raft_consensus.cc:2781] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:09:17.686304 11202 raft_consensus.cc:482] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:09:17.686348 11202 raft_consensus.cc:3037] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:09:17.687391 11202 raft_consensus.cc:504] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:17.687778 11202 leader_election.cc:288] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd [CANDIDATE]: Term 1 election: Requested vote from peers 40237e8dc68b4fc6ada088cfba203a52 (127.27.225.194:43135), 04cf9ddb3b194f778e793f21766db37d (127.27.225.193:35613)
I20260806 21:09:17.688186 11001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "586b330667c04e128907ff1fada91fd6" candidate_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40237e8dc68b4fc6ada088cfba203a52"
I20260806 21:09:17.688293 10859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "586b330667c04e128907ff1fada91fd6" candidate_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04cf9ddb3b194f778e793f21766db37d"
I20260806 21:09:17.688313 11001 raft_consensus.cc:3037] T 586b330667c04e128907ff1fada91fd6 P 40237e8dc68b4fc6ada088cfba203a52 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:09:17.688458 10859 raft_consensus.cc:3037] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:09:17.689504 11001 raft_consensus.cc:2454] T 586b330667c04e128907ff1fada91fd6 P 40237e8dc68b4fc6ada088cfba203a52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44df30bca10d4c36a0d1c41f8bff84dd in term 1.
I20260806 21:09:17.689507 10859 raft_consensus.cc:2454] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44df30bca10d4c36a0d1c41f8bff84dd in term 1.
I20260806 21:09:17.689837 11068 leader_election.cc:302] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40237e8dc68b4fc6ada088cfba203a52, 44df30bca10d4c36a0d1c41f8bff84dd; no voters:
I20260806 21:09:17.690050 11202 raft_consensus.cc:2781] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:09:17.690326 11202 raft_consensus.cc:686] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd [term 1 LEADER]: Becoming Leader. State: Replica: 44df30bca10d4c36a0d1c41f8bff84dd, State: Running, Role: LEADER
I20260806 21:09:17.690723 11202 consensus_queue.cc:237] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:17.694293 10715 catalog_manager.cc:5495] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 44df30bca10d4c36a0d1c41f8bff84dd (127.27.225.195). New cstate: current_term: 1 leader_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } health_report { overall_health: UNKNOWN } } }
W20260806 21:09:17.700940 11185 tablet.cc:2369] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 21:09:17.736730 11049 tablet.cc:2369] T 586b330667c04e128907ff1fada91fd6 P 40237e8dc68b4fc6ada088cfba203a52: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:09:17.741451 11195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:17.737348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39298 (local address 127.27.225.195:38791)
0806 21:09:17.737495 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:17.737499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:17.737513 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:09:17.737619 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:17.737622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:17.737677 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:17.737768 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:09:17.738178 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.738704 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.739561 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:17.739813 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:17.739908 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:17.740351 (+ 443us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:09:17.740863 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:09:17.740978 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:17.741086 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 21:09:17.741195 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 21:09:17.747954 10859 raft_consensus.cc:1261] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [term 1 FOLLOWER]: Refusing update from remote peer 44df30bca10d4c36a0d1c41f8bff84dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:09:17.747957 11001 raft_consensus.cc:1261] T 586b330667c04e128907ff1fada91fd6 P 40237e8dc68b4fc6ada088cfba203a52 [term 1 FOLLOWER]: Refusing update from remote peer 44df30bca10d4c36a0d1c41f8bff84dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:09:17.748840 11206 consensus_queue.cc:1035] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:09:17.749153 11202 consensus_queue.cc:1035] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:09:17.756165 11209 mvcc.cc:205] Tried to move back new op lower bound from 7315663084527828992 to 7315663084304351232. Current Snapshot: MvccSnapshot[applied={T|T < 7315663084527828992}]
I20260806 21:09:17.764596 10714 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:51744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:09:17.764849 10714 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:51744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:09:17.767498 10714 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38: Sending DeleteTablet for 3 replicas of tablet 586b330667c04e128907ff1fada91fd6
I20260806 21:09:17.768308 10980 tablet_service.cc:1514] Processing DeleteTablet for tablet 586b330667c04e128907ff1fada91fd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:09:17 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:46726
I20260806 21:09:17.768383 11116 tablet_service.cc:1514] Processing DeleteTablet for tablet 586b330667c04e128907ff1fada91fd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:09:17 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:39284
I20260806 21:09:17.768504 10838 tablet_service.cc:1514] Processing DeleteTablet for tablet 586b330667c04e128907ff1fada91fd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:09:17 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:42490
I20260806 21:09:17.768906 11219 tablet_replica.cc:330] T 586b330667c04e128907ff1fada91fd6 P 40237e8dc68b4fc6ada088cfba203a52: stopping tablet replica
I20260806 21:09:17.769066 11220 tablet_replica.cc:330] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd: stopping tablet replica
I20260806 21:09:17.769114 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 10753
I20260806 21:09:17.769304 11219 raft_consensus.cc:2229] T 586b330667c04e128907ff1fada91fd6 P 40237e8dc68b4fc6ada088cfba203a52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:09:17.769357 11220 raft_consensus.cc:2229] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:09:17.769647 11219 raft_consensus.cc:2258] T 586b330667c04e128907ff1fada91fd6 P 40237e8dc68b4fc6ada088cfba203a52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:09:17.769742 11220 raft_consensus.cc:2258] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:09:17.771988 11219 ts_tablet_manager.cc:1905] T 586b330667c04e128907ff1fada91fd6 P 40237e8dc68b4fc6ada088cfba203a52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:09:17.775002 11219 ts_tablet_manager.cc:1918] T 586b330667c04e128907ff1fada91fd6 P 40237e8dc68b4fc6ada088cfba203a52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:09:17.775118 11219 log.cc:1196] T 586b330667c04e128907ff1fada91fd6 P 40237e8dc68b4fc6ada088cfba203a52: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/wal/wals/586b330667c04e128907ff1fada91fd6
I20260806 21:09:17.775446 11219 ts_tablet_manager.cc:1939] T 586b330667c04e128907ff1fada91fd6 P 40237e8dc68b4fc6ada088cfba203a52: Deleting consensus metadata
I20260806 21:09:17.776921 10697 catalog_manager.cc:4841] TS 40237e8dc68b4fc6ada088cfba203a52 (127.27.225.194:43135): tablet 586b330667c04e128907ff1fada91fd6 (table test-table [id=af7ead9dadd64c1d9afd87ea6892a439]) successfully deleted
I20260806 21:09:17.777730 11220 ts_tablet_manager.cc:1905] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 21:09:17.779345 10698 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.225.193:35613 (error 108)
I20260806 21:09:17.780370 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 10910
I20260806 21:09:17.780560 10913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:17.780160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:35613 (local address 127.0.0.1:42498)
0806 21:09:17.780359 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:17.780453 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.193:35613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 21:09:17.780642 11220 ts_tablet_manager.cc:1918] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:09:17.780748 11220 log.cc:1196] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/586b330667c04e128907ff1fada91fd6
I20260806 21:09:17.781119 11220 ts_tablet_manager.cc:1939] T 586b330667c04e128907ff1fada91fd6 P 44df30bca10d4c36a0d1c41f8bff84dd: Deleting consensus metadata
I20260806 21:09:17.782402 10701 catalog_manager.cc:4841] TS 44df30bca10d4c36a0d1c41f8bff84dd (127.27.225.195:38791): tablet 586b330667c04e128907ff1fada91fd6 (table test-table [id=af7ead9dadd64c1d9afd87ea6892a439]) successfully deleted
W20260806 21:09:17.781766 10698 catalog_manager.cc:4584] TS 04cf9ddb3b194f778e793f21766db37d (127.27.225.193:35613): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 586b330667c04e128907ff1fada91fd6: Network error: Client connection negotiation failed: client connection to 127.27.225.193:35613: connect: Connection refused (error 111)
I20260806 21:09:17.793211 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 11051
I20260806 21:09:17.799113 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 10679
I20260806 21:09:17.807454 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:40157
--webserver_interface=127.27.225.254
--webserver_port=37955
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:36065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:40157
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:09:17.931736 11222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:17.932045 11222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:17.932089 11222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:17.936295 11222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:09:17.936376 11222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:17.936398 11222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:17.936414 11222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:09:17.936430 11222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:09:17.942178 11222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:36065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:40157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:40157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=37955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:17.943332 11222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:17.944361 11222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:17.953603 11229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:17.953569 11231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:17.953613 11227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:17.954007 11222 server_base.cc:1034] running on GCE node
I20260806 21:09:17.954589 11222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:17.955897 11222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:17.957111 11222 hybrid_clock.cc:648] HybridClock initialized: now 1786050557957095 us; error 45 us; skew 500 ppm
Aug 06 21:09:17 dist-test-slave-69rg krb5kdc[10114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050557, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:17.960178 11222 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:09:17.961136 11222 webserver.cc:460] Webserver started at http://127.27.225.254:37955/ using document root <none> and password file <none>
I20260806 21:09:17.961751 11222 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:17.961877 11222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:17.966110 11222 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 21:09:17.968447 11238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:17.969880 11222 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:09:17.970050 11222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "a8326cac2b564a289b3a80bca33e7d38"
format_stamp: "Formatted at 2026-08-06 21:09:15 on dist-test-slave-69rg"
server_key: "1f1319a8ecda1ae6596ac9cad82df334"
server_key_iv: "d33404251e8182f753e8bb88219dcb48"
server_key_version: "encryptionkey@0"
I20260806 21:09:17.970674 11222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:17.987402 11222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:17.990653 11222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:17.990880 11222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:17.998591 11222 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:40157
I20260806 21:09:17.998920 11292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:40157 every 8 connection(s)
I20260806 21:09:17.999869 11222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:09:18.004712 11293 sys_catalog.cc:263] Verifying existing consensus state
I20260806 21:09:18.006289 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 11222
I20260806 21:09:18.006834 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:35613
--local_ip_for_outbound_sockets=127.27.225.193
--tserver_master_addrs=127.27.225.254:40157
--webserver_port=35679
--webserver_interface=127.27.225.193
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:36065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:09:18.006910 11293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38: Bootstrap starting.
I20260806 21:09:18.027750 11293 log.cc:824] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38: Log is configured to *not* fsync() on all Append() calls
I20260806 21:09:18.039335 11293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 21:09:18.039786 11293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38: Bootstrap complete.
I20260806 21:09:18.044744 11293 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8326cac2b564a289b3a80bca33e7d38" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40157 } }
I20260806 21:09:18.045233 11293 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8326cac2b564a289b3a80bca33e7d38, State: Initialized, Role: FOLLOWER
I20260806 21:09:18.045781 11293 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8326cac2b564a289b3a80bca33e7d38" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40157 } }
I20260806 21:09:18.046000 11293 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:09:18.046101 11293 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:09:18.046233 11293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [term 2 FOLLOWER]: Advancing to term 3
I20260806 21:09:18.048488 11293 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8326cac2b564a289b3a80bca33e7d38" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40157 } }
I20260806 21:09:18.048880 11293 leader_election.cc:302] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8326cac2b564a289b3a80bca33e7d38; no voters:
I20260806 21:09:18.049297 11293 leader_election.cc:288] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 21:09:18.049388 11298 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 21:09:18.049585 11298 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [term 3 LEADER]: Becoming Leader. State: Replica: a8326cac2b564a289b3a80bca33e7d38, State: Running, Role: LEADER
I20260806 21:09:18.049961 11298 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8326cac2b564a289b3a80bca33e7d38" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40157 } }
I20260806 21:09:18.050686 11293 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:09:18.051560 11299 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a8326cac2b564a289b3a80bca33e7d38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8326cac2b564a289b3a80bca33e7d38" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40157 } } }
I20260806 21:09:18.051707 11299 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [sys.catalog]: This master's current role is: LEADER
I20260806 21:09:18.052080 11300 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8326cac2b564a289b3a80bca33e7d38. Latest consensus state: current_term: 3 leader_uuid: "a8326cac2b564a289b3a80bca33e7d38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8326cac2b564a289b3a80bca33e7d38" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40157 } } }
I20260806 21:09:18.052238 11300 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8326cac2b564a289b3a80bca33e7d38 [sys.catalog]: This master's current role is: LEADER
I20260806 21:09:18.052773 11308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:09:18.059535 11308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:09:18.060755 11308 catalog_manager.cc:1259] Loaded cluster ID: f62d58e3dd3441cd86ccc5ffed368f7a
I20260806 21:09:18.060979 11308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 21:09:18.063071 11308 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f33a49079ad google::LogMessage::Fail() at ??:0
@ 0x7f33a490a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f33a49074de google::LogMessage::Flush() at ??:0
@ 0x7f33a4908a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f33af246fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f33af247d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f33af244579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f33af288640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f33a6b593d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f33a6b59c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f33a6b5b3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f33a6b4a512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f33a818e6db start_thread at ??:0
@ 0x7f33a340561f clone at ??:0
W20260806 21:09:18.171120 11295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:18.171473 11295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:18.171531 11295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:18.175940 11295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:18.176023 11295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:18.176095 11295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:09:18.182086 11295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:36065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:35613
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=35679
--tserver_master_addrs=127.27.225.254:40157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:18.183300 11295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:18.184397 11295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:18.194891 11338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:18.194972 11340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:18.194896 11337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:18.195528 11295 server_base.cc:1034] running on GCE node
I20260806 21:09:18.195987 11295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:18.196797 11295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:18.197989 11295 hybrid_clock.cc:648] HybridClock initialized: now 1786050558197966 us; error 39 us; skew 500 ppm
Aug 06 21:09:18 dist-test-slave-69rg krb5kdc[10114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050558, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:18.201472 11295 init.cc:377] Logged in from keytab as kudu/127.27.225.193@KRBTEST.COM (short username kudu)
I20260806 21:09:18.202629 11295 webserver.cc:460] Webserver started at http://127.27.225.193:35679/ using document root <none> and password file <none>
I20260806 21:09:18.203291 11295 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:18.203408 11295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:18.207582 11295 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 21:09:18.210018 11347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:18.211424 11295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:09:18.211572 11295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "04cf9ddb3b194f778e793f21766db37d"
format_stamp: "Formatted at 2026-08-06 21:09:15 on dist-test-slave-69rg"
server_key: "424bbe2cb4e5e30ee84733d739b134a5"
server_key_iv: "b73fec191d17567b21271906298cc16a"
server_key_version: "encryptionkey@0"
I20260806 21:09:18.212152 11295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:18.229303 11295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:18.232789 11295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:18.233027 11295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:18.233688 11295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:09:18.234962 11354 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 21:09:18.240877 11295 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 21:09:18.240984 11295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260806 21:09:18.241076 11295 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 21:09:18.242909 11295 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 21:09:18.242988 11295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.001s
I20260806 21:09:18.243341 11354 tablet_bootstrap.cc:492] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d: Bootstrap starting.
I20260806 21:09:18.254099 11295 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:35613
I20260806 21:09:18.254385 11465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:35613 every 8 connection(s)
I20260806 21:09:18.255187 11295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:09:18.257682 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 11295
I20260806 21:09:18.258215 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:43135
--local_ip_for_outbound_sockets=127.27.225.194
--tserver_master_addrs=127.27.225.254:40157
--webserver_port=38499
--webserver_interface=127.27.225.194
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:36065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:09:18.265734 11354 log.cc:824] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d: Log is configured to *not* fsync() on all Append() calls
I20260806 21:09:18.275846 11354 tablet_bootstrap.cc:492] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 21:09:18.276273 11354 tablet_bootstrap.cc:492] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d: Bootstrap complete.
I20260806 21:09:18.277410 11354 ts_tablet_manager.cc:1397] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d: Time spent bootstrapping tablet: real 0.034s user 0.025s sys 0.002s
I20260806 21:09:18.280185 11354 raft_consensus.cc:348] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:18.280710 11354 raft_consensus.cc:729] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04cf9ddb3b194f778e793f21766db37d, State: Initialized, Role: FOLLOWER
I20260806 21:09:18.281250 11354 consensus_queue.cc:260] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:18.282339 11354 ts_tablet_manager.cc:1428] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
W20260806 21:09:18.311204 11468 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:09:18.257246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.193:45845)
0806 21:09:18.258834 (+ 1588us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:18.258903 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:18.260199 (+ 1296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:18.310440 (+ 50241us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1329,"thread_start_us":197,"threads_started":1}
W20260806 21:09:18.311885 11466 heartbeater.cc:644] Failed to heartbeat to 127.27.225.254:40157 (0 consecutive failures): Network error: Failed to ping master at 127.27.225.254:40157: Client connection negotiation failed: client connection to 127.27.225.254:40157: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 21:09:18.312705 11468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:18.312412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.193:56861)
0806 21:09:18.312555 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:18.312640 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 21:09:18.313556 11468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:18.313321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.193:41243)
0806 21:09:18.313445 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:18.313503 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 21:09:18.313774 11466 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 21:09:18.393646 11469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:18.393982 11469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:18.394032 11469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:18.398329 11469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:18.398420 11469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:18.398485 11469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:09:18.404369 11469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:36065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:43135
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=38499
--tserver_master_addrs=127.27.225.254:40157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:18.405411 11469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:18.406454 11469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:18.416325 11476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:18.416370 11477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:18.416370 11479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:18.416601 11469 server_base.cc:1034] running on GCE node
I20260806 21:09:18.417099 11469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:18.417779 11469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:18.419035 11469 hybrid_clock.cc:648] HybridClock initialized: now 1786050558418998 us; error 46 us; skew 500 ppm
Aug 06 21:09:18 dist-test-slave-69rg krb5kdc[10114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050558, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:18.422034 11469 init.cc:377] Logged in from keytab as kudu/127.27.225.194@KRBTEST.COM (short username kudu)
I20260806 21:09:18.423053 11469 webserver.cc:460] Webserver started at http://127.27.225.194:38499/ using document root <none> and password file <none>
I20260806 21:09:18.423651 11469 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:18.423756 11469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:18.428287 11469 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 21:09:18.430478 11486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:18.432176 11469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:09:18.432349 11469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "40237e8dc68b4fc6ada088cfba203a52"
format_stamp: "Formatted at 2026-08-06 21:09:16 on dist-test-slave-69rg"
server_key: "ea5909bb6f9227fdd8115e213b96bb99"
server_key_iv: "c06b568a59b8c2d4c779ef16b9d924fe"
server_key_version: "encryptionkey@0"
I20260806 21:09:18.432971 11469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:18.447950 11469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:18.451589 11469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:18.451844 11469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:18.452507 11469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:09:18.453387 11469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:09:18.453461 11469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:18.453533 11469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:09:18.453583 11469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:18.462949 11469 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:43135
I20260806 21:09:18.462981 11606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:43135 every 8 connection(s)
I20260806 21:09:18.463959 11469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:09:18.467999 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:18.466136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.194:56613)
0806 21:09:18.466615 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:18.466736 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":118,"threads_started":1}
W20260806 21:09:18.468495 11607 heartbeater.cc:644] Failed to heartbeat to 127.27.225.254:40157 (0 consecutive failures): Network error: Failed to ping master at 127.27.225.254:40157: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
I20260806 21:09:18.469269 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:18.468994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.194:47923)
0806 21:09:18.469131 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:18.469193 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 21:09:18.469931 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:18.469678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.194:49535)
0806 21:09:18.469793 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:18.469865 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 21:09:18.470108 11607 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 21:09:18.472362 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 11469
I20260806 21:09:18.472810 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:38791
--local_ip_for_outbound_sockets=127.27.225.195
--tserver_master_addrs=127.27.225.254:40157
--webserver_port=39733
--webserver_interface=127.27.225.195
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:36065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:09:18.590178 11610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:18.590474 11610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:18.590520 11610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:18.594586 11610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:18.594651 11610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:18.594710 11610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:09:18.600430 11610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:36065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:38791
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=39733
--tserver_master_addrs=127.27.225.254:40157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:18.601660 11610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:18.602780 11610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:18.613162 11616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:18.613374 11610 server_base.cc:1034] running on GCE node
W20260806 21:09:18.613260 11615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:18.613217 11618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:18.614037 11610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:18.614722 11610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:18.615909 11610 hybrid_clock.cc:648] HybridClock initialized: now 1786050558615880 us; error 55 us; skew 500 ppm
Aug 06 21:09:18 dist-test-slave-69rg krb5kdc[10114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050558, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:18.618953 11610 init.cc:377] Logged in from keytab as kudu/127.27.225.195@KRBTEST.COM (short username kudu)
I20260806 21:09:18.619900 11610 webserver.cc:460] Webserver started at http://127.27.225.195:39733/ using document root <none> and password file <none>
I20260806 21:09:18.620494 11610 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:18.620559 11610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:18.624698 11610 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:09:18.626968 11625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:18.628316 11610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:09:18.628468 11610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "44df30bca10d4c36a0d1c41f8bff84dd"
format_stamp: "Formatted at 2026-08-06 21:09:16 on dist-test-slave-69rg"
server_key: "3e56eb8ad2b5391f3e8800b67317299c"
server_key_iv: "daed8bcf27530384a3495a5fc8ad3de4"
server_key_version: "encryptionkey@0"
I20260806 21:09:18.629051 11610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:18.644162 11610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:18.648162 11610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:18.648429 11610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:18.649145 11610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:09:18.650228 11610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:09:18.650313 11610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:18.650390 11610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:09:18.650444 11610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:18.661137 11610 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:38791
I20260806 21:09:18.661192 11741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:38791 every 8 connection(s)
I20260806 21:09:18.662246 11610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:09:18.666451 11744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:18.664568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.195:35945)
0806 21:09:18.665042 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:18.665160 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":136,"threads_started":1}
W20260806 21:09:18.667007 11742 heartbeater.cc:644] Failed to heartbeat to 127.27.225.254:40157 (0 consecutive failures): Network error: Failed to ping master at 127.27.225.254:40157: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
I20260806 21:09:18.667634 11744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:18.667410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.195:32935)
0806 21:09:18.667524 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:18.667575 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 21:09:18.668270 11744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:18.668074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.195:33645)
0806 21:09:18.668173 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:18.668214 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 21:09:18.668486 11742 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 21:09:18.671183 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 11610
I20260806 21:09:19.315042 11746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:19.314540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.193:46835)
0806 21:09:19.314855 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:19.314921 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":85,"threads_started":1}
I20260806 21:09:19.471210 11747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:19.470780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.194:44301)
0806 21:09:19.471039 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:19.471101 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":75,"threads_started":1}
I20260806 21:09:19.669560 11748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:19.669103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.195:53791)
0806 21:09:19.669385 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:19.669446 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":87,"threads_started":1}
I20260806 21:09:19.826432 11749 raft_consensus.cc:482] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:09:19.826663 11749 raft_consensus.cc:504] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:19.827791 11749 leader_election.cc:288] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 40237e8dc68b4fc6ada088cfba203a52 (127.27.225.194:43135), 44df30bca10d4c36a0d1c41f8bff84dd (127.27.225.195:38791)
Aug 06 21:09:19 dist-test-slave-69rg krb5kdc[10114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050558, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for kudu/127.27.225.194@KRBTEST.COM
Aug 06 21:09:19 dist-test-slave-69rg krb5kdc[10114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050558, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for kudu/127.27.225.195@KRBTEST.COM
I20260806 21:09:19.843438 11752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:19.828242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:40727 (local address 127.27.225.194:43135)
0806 21:09:19.828724 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:19.828746 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:19.828782 (+ 36us) server_negotiation.cc:407] Connection header received
0806 21:09:19.829482 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:19.829487 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:19.829903 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:19.830231 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:19.831462 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:19.832737 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:19.834190 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:19.834719 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:19.834916 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:19.838495 (+ 3579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:19.838507 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:19.838528 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:19.838571 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:19.840588 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:19.841233 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:19.841239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:19.841245 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:19.841376 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:19.841739 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:19.841743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:19.841745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:19.842257 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:19.842408 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:19.842958 (+ 550us) server_negotiation.cc:299] Negotiation successful
0806 21:09:19.843250 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":65,"threads_started":1}
I20260806 21:09:19.843422 11750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:19.828036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:43135 (local address 127.27.225.193:40727)
0806 21:09:19.828363 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:19.828383 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:19.828491 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:19.830283 (+ 1792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:19.830289 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:19.830823 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:19.831269 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:19.831285 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:19.832924 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:19.832927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:19.834058 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:19.834069 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:19.834875 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:19.834879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:19.835128 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:19.835850 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:19.835866 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:19.838072 (+ 2206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:19.840759 (+ 2687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:19.840765 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:19.840780 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:19.841097 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:19.841480 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:19.841485 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:19.841487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:19.841639 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:19.842382 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:19.842389 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:19.842638 (+ 249us) client_negotiation.cc:769] Sending connection context
0806 21:09:19.842848 (+ 210us) client_negotiation.cc:240] Negotiation successful
0806 21:09:19.843225 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":158,"spinlock_wait_cycles":25728,"thread_start_us":71,"threads_started":1}
I20260806 21:09:19.844200 11751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:19.828036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:38791 (local address 127.27.225.193:57975)
0806 21:09:19.828481 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:19.828500 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:19.828623 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:19.830285 (+ 1662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:19.830291 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:19.830823 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:19.831270 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:19.831287 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:19.832994 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:19.833000 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:19.834059 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:19.834070 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:19.834856 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:19.834860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:19.835108 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:19.835913 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:19.835926 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:19.838436 (+ 2510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:19.840934 (+ 2498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:19.840941 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:19.840943 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:19.841201 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:19.841628 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:19.841632 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:19.841634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:19.841733 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:19.842421 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:19.842425 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:19.842633 (+ 208us) client_negotiation.cc:769] Sending connection context
0806 21:09:19.842848 (+ 215us) client_negotiation.cc:240] Negotiation successful
0806 21:09:19.843340 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":287,"spinlock_wait_cycles":11264,"thread_start_us":66,"threads_started":1}
I20260806 21:09:19.844950 11753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:19.828257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:57975 (local address 127.27.225.195:38791)
0806 21:09:19.828732 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:19.828760 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:19.828793 (+ 33us) server_negotiation.cc:407] Connection header received
0806 21:09:19.829480 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:19.829486 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:19.829907 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:19.830231 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:19.831483 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:19.832868 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:19.834223 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:19.834717 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:19.834895 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:19.838664 (+ 3769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:19.838670 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:19.838683 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:19.838720 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:19.840793 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:19.841342 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:19.841347 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:19.841351 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:19.841504 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:19.841885 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:19.841888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:19.841890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:19.842295 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:19.842456 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:19.844342 (+ 1886us) server_negotiation.cc:299] Negotiation successful
0806 21:09:19.844759 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":118,"threads_started":1}
W20260806 21:09:19.846511 11348 leader_election.cc:341] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 40237e8dc68b4fc6ada088cfba203a52 (127.27.225.194:43135): Not found: Tablet not found: 586b330667c04e128907ff1fada91fd6
W20260806 21:09:19.847070 11351 leader_election.cc:341] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 44df30bca10d4c36a0d1c41f8bff84dd (127.27.225.195:38791): Not found: Tablet not found: 586b330667c04e128907ff1fada91fd6
I20260806 21:09:19.847149 11351 leader_election.cc:302] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 04cf9ddb3b194f778e793f21766db37d; no voters: 40237e8dc68b4fc6ada088cfba203a52, 44df30bca10d4c36a0d1c41f8bff84dd
I20260806 21:09:19.847404 11749 raft_consensus.cc:2726] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 21:09:20.316450 11751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:20.316119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.193:48369)
0806 21:09:20.316272 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:20.316356 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 21:09:20.472486 11754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:20.472035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.194:58611)
0806 21:09:20.472325 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:20.472386 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":85,"threads_started":1}
I20260806 21:09:20.671291 11755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:20.670682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.195:45045)
0806 21:09:20.671015 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:20.671078 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":83,"threads_started":1}
I20260806 21:09:21.317929 11756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:21.317397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.193:48743)
0806 21:09:21.317695 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:21.317758 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":84,"threads_started":1}
I20260806 21:09:21.473955 11757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:21.473389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.194:60785)
0806 21:09:21.473705 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:21.473769 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":84,"threads_started":1}
I20260806 21:09:21.672756 11758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:21.672223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.195:39971)
0806 21:09:21.672542 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:21.672622 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":97,"threads_started":1}
I20260806 21:09:21.968562 11759 raft_consensus.cc:482] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:09:21.968734 11759 raft_consensus.cc:504] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:21.969239 11759 leader_election.cc:288] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 40237e8dc68b4fc6ada088cfba203a52 (127.27.225.194:43135), 44df30bca10d4c36a0d1c41f8bff84dd (127.27.225.195:38791)
W20260806 21:09:21.970149 11348 leader_election.cc:341] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 40237e8dc68b4fc6ada088cfba203a52 (127.27.225.194:43135): Not found: Tablet not found: 586b330667c04e128907ff1fada91fd6
W20260806 21:09:21.970258 11351 leader_election.cc:341] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 44df30bca10d4c36a0d1c41f8bff84dd (127.27.225.195:38791): Not found: Tablet not found: 586b330667c04e128907ff1fada91fd6
I20260806 21:09:21.970345 11351 leader_election.cc:302] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 04cf9ddb3b194f778e793f21766db37d; no voters: 40237e8dc68b4fc6ada088cfba203a52, 44df30bca10d4c36a0d1c41f8bff84dd
I20260806 21:09:21.970499 11759 raft_consensus.cc:2726] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 21:09:22.319268 11760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:22.318835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.193:42473)
0806 21:09:22.319083 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:22.319136 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":79,"threads_started":1}
I20260806 21:09:22.475132 11761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:22.474731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.194:45749)
0806 21:09:22.474980 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:22.475036 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":83,"threads_started":1}
I20260806 21:09:22.674201 11762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:22.673687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.195:42319)
0806 21:09:22.674035 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:22.674100 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":81,"threads_started":1}
I20260806 21:09:23.321035 11763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:23.320470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.193:53085)
0806 21:09:23.320851 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:23.320923 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":94,"threads_started":1}
I20260806 21:09:23.476508 11765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:23.476052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.194:33177)
0806 21:09:23.476338 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:23.476401 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":98,"threads_started":1}
I20260806 21:09:23.675705 11766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:23.675189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.195:49207)
0806 21:09:23.675518 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:23.675604 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":126,"threads_started":1}
I20260806 21:09:24.322481 11767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:24.322027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.193:46753)
0806 21:09:24.322303 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:24.322377 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":96,"threads_started":1}
I20260806 21:09:24.448378 11769 raft_consensus.cc:482] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:09:24.448523 11769 raft_consensus.cc:504] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:24.448958 11769 leader_election.cc:288] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 40237e8dc68b4fc6ada088cfba203a52 (127.27.225.194:43135), 44df30bca10d4c36a0d1c41f8bff84dd (127.27.225.195:38791)
W20260806 21:09:24.449787 11348 leader_election.cc:341] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 40237e8dc68b4fc6ada088cfba203a52 (127.27.225.194:43135): Not found: Tablet not found: 586b330667c04e128907ff1fada91fd6
W20260806 21:09:24.449966 11351 leader_election.cc:341] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 44df30bca10d4c36a0d1c41f8bff84dd (127.27.225.195:38791): Not found: Tablet not found: 586b330667c04e128907ff1fada91fd6
I20260806 21:09:24.450053 11351 leader_election.cc:302] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 04cf9ddb3b194f778e793f21766db37d; no voters: 40237e8dc68b4fc6ada088cfba203a52, 44df30bca10d4c36a0d1c41f8bff84dd
I20260806 21:09:24.450212 11769 raft_consensus.cc:2726] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 21:09:24.477648 11770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:24.477244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.194:52059)
0806 21:09:24.477503 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:24.477556 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":75,"threads_started":1}
I20260806 21:09:24.677109 11771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:24.676647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.195:60701)
0806 21:09:24.676951 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:24.677011 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":103,"threads_started":1}
I20260806 21:09:25.323765 11772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:25.323336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.193:46665)
0806 21:09:25.323606 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:25.323667 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":78,"threads_started":1}
I20260806 21:09:25.479023 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:25.478556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.194:55477)
0806 21:09:25.478867 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:25.478928 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":77,"threads_started":1}
I20260806 21:09:25.678845 11774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:25.678200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.195:49487)
0806 21:09:25.678579 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:25.678715 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":102,"threads_started":1}
I20260806 21:09:26.325016 11775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:26.324588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.193:55361)
0806 21:09:26.324860 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:26.324919 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":74,"threads_started":1}
I20260806 21:09:26.480178 11776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:26.479764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.194:37697)
0806 21:09:26.480023 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:26.480083 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":77,"threads_started":1}
I20260806 21:09:26.680261 11777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:26.679727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.195:56035)
0806 21:09:26.680059 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:26.680134 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":97,"threads_started":1}
I20260806 21:09:26.915371 11778 raft_consensus.cc:482] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:09:26.915545 11778 raft_consensus.cc:504] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:26.916066 11778 leader_election.cc:288] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 40237e8dc68b4fc6ada088cfba203a52 (127.27.225.194:43135), 44df30bca10d4c36a0d1c41f8bff84dd (127.27.225.195:38791)
W20260806 21:09:26.916786 11348 leader_election.cc:341] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 40237e8dc68b4fc6ada088cfba203a52 (127.27.225.194:43135): Not found: Tablet not found: 586b330667c04e128907ff1fada91fd6
W20260806 21:09:26.916924 11351 leader_election.cc:341] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 44df30bca10d4c36a0d1c41f8bff84dd (127.27.225.195:38791): Not found: Tablet not found: 586b330667c04e128907ff1fada91fd6
I20260806 21:09:26.917040 11351 leader_election.cc:302] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 04cf9ddb3b194f778e793f21766db37d; no voters: 40237e8dc68b4fc6ada088cfba203a52, 44df30bca10d4c36a0d1c41f8bff84dd
I20260806 21:09:26.917181 11778 raft_consensus.cc:2726] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 21:09:27.326351 11779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:27.325876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.193:39091)
0806 21:09:27.326181 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:27.326253 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":92,"threads_started":1}
I20260806 21:09:27.481503 11780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:27.480973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.194:34931)
0806 21:09:27.481322 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:27.481389 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":101,"threads_started":1}
I20260806 21:09:27.681692 11781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:27.681179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.195:47453)
0806 21:09:27.681505 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:27.681572 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":102,"threads_started":1}
I20260806 21:09:28.327989 11782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:28.327371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.193:48255)
0806 21:09:28.327801 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:28.327879 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":84,"threads_started":1}
I20260806 21:09:28.483033 11783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:28.482558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.194:51193)
0806 21:09:28.482863 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:28.482937 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":112,"threads_started":1}
I20260806 21:09:28.683243 11784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:28.682790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.195:35179)
0806 21:09:28.683075 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:28.683136 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":98,"threads_started":1}
I20260806 21:09:29.329406 11785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:29.328940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.193:54723)
0806 21:09:29.329241 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:29.329306 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":83,"threads_started":1}
I20260806 21:09:29.484304 11786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:29.483848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.194:42965)
0806 21:09:29.484132 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:29.484197 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":76,"threads_started":1}
I20260806 21:09:29.684696 11787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:29.684208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.195:55241)
0806 21:09:29.684528 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:29.684594 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":80,"threads_started":1}
I20260806 21:09:30.330819 11788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:30.330321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.193:53273)
0806 21:09:30.330609 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:30.330677 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":116,"threads_started":1}
I20260806 21:09:30.485807 11789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:30.485324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.194:55469)
0806 21:09:30.485647 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:30.485711 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":91,"threads_started":1}
I20260806 21:09:30.686002 11790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:30.685540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.195:50715)
0806 21:09:30.685841 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:30.685903 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":81,"threads_started":1}
I20260806 21:09:30.985486 11791 raft_consensus.cc:482] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:09:30.985706 11791 raft_consensus.cc:504] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40237e8dc68b4fc6ada088cfba203a52" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43135 } } peers { permanent_uuid: "44df30bca10d4c36a0d1c41f8bff84dd" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38791 } } peers { permanent_uuid: "04cf9ddb3b194f778e793f21766db37d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 35613 } }
I20260806 21:09:30.986300 11791 leader_election.cc:288] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 40237e8dc68b4fc6ada088cfba203a52 (127.27.225.194:43135), 44df30bca10d4c36a0d1c41f8bff84dd (127.27.225.195:38791)
W20260806 21:09:30.987077 11348 leader_election.cc:341] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 40237e8dc68b4fc6ada088cfba203a52 (127.27.225.194:43135): Not found: Tablet not found: 586b330667c04e128907ff1fada91fd6
W20260806 21:09:30.987185 11351 leader_election.cc:341] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 44df30bca10d4c36a0d1c41f8bff84dd (127.27.225.195:38791): Not found: Tablet not found: 586b330667c04e128907ff1fada91fd6
I20260806 21:09:30.987303 11351 leader_election.cc:302] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 04cf9ddb3b194f778e793f21766db37d; no voters: 40237e8dc68b4fc6ada088cfba203a52, 44df30bca10d4c36a0d1c41f8bff84dd
I20260806 21:09:30.987471 11791 raft_consensus.cc:2726] T 586b330667c04e128907ff1fada91fd6 P 04cf9ddb3b194f778e793f21766db37d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 21:09:31.332226 11792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:31.331681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.193:44763)
0806 21:09:31.331987 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:31.332052 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":94,"threads_started":1}
I20260806 21:09:31.487141 11793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:31.486704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.194:57087)
0806 21:09:31.486960 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:31.487021 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":81,"threads_started":1}
I20260806 21:09:31.687261 11794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:31.686777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.195:57095)
0806 21:09:31.687057 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:31.687163 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":124,"threads_started":1}
I20260806 21:09:32.333530 11795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:32.333008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.193:39669)
0806 21:09:32.333288 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:32.333360 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":79,"threads_started":1}
I20260806 21:09:32.488497 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:32.488005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.194:39131)
0806 21:09:32.488330 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:32.488395 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":91,"threads_started":1}
I20260806 21:09:32.688679 11797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:32.688169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.195:45167)
0806 21:09:32.688498 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:32.688561 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":130,"threads_started":1}
I20260806 21:09:33.335083 11798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:33.334636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.193:47125)
0806 21:09:33.334928 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:33.334984 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":90,"threads_started":1}
I20260806 21:09:33.489938 11799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:33.489428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:40157 (local address 127.27.225.194:50755)
0806 21:09:33.489696 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:33.489751 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:40157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":77,"threads_started":1}
I20260806 21:09:33.675175 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 11295
I20260806 21:09:33.682260 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 11469
I20260806 21:09:33.689059 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 11610
2026-08-06T21:09:33Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19732 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:09:33 dist-test-slave-69rg krb5kdc[11803](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:09:33 dist-test-slave-69rg krb5kdc[11803](info): set up 2 sockets
Aug 06 21:09:33 dist-test-slave-69rg krb5kdc[11803](info): commencing operation
krb5kdc: starting...
W20260806 21:09:34.745738 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:09:34 dist-test-slave-69rg krb5kdc[11803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:09:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:09:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:09:34.883630 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:41497
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:40747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:41497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:09:35.002576 11816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:35.003001 11816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:35.003342 11816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:35.007846 11816 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 21:09:35.007979 11816 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 21:09:35.008005 11816 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 21:09:35.008028 11816 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:09:35.008073 11816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:35.008117 11816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:35.008159 11816 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:09:35.008198 11816 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:09:35.017206 11816 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:40747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:41497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:41497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:35.019605 11816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:35.021112 11816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:09:35.032326 11816 server_base.cc:1034] running on GCE node
W20260806 21:09:35.032279 11821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:35.032289 11822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:35.032280 11824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:35.033514 11816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:35.034940 11816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:35.036299 11816 hybrid_clock.cc:648] HybridClock initialized: now 1786050575036262 us; error 75 us; skew 500 ppm
Aug 06 21:09:35 dist-test-slave-69rg krb5kdc[11803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050575, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:35.040241 11816 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:09:35.041476 11816 webserver.cc:460] Webserver started at http://127.27.225.254:38293/ using document root <none> and password file <none>
I20260806 21:09:35.042318 11816 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:35.042446 11816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:35.042721 11816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:09:35.045121 11816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "387028fb73dc4583ad3c8fd99446462d"
format_stamp: "Formatted at 2026-08-06 21:09:35 on dist-test-slave-69rg"
server_key: "6b06e1265b88f856ac7443f108d68a25"
server_key_iv: "4cf774346be6934574001de52da924a8"
server_key_version: "encryptionkey@0"
I20260806 21:09:35.046130 11816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "387028fb73dc4583ad3c8fd99446462d"
format_stamp: "Formatted at 2026-08-06 21:09:35 on dist-test-slave-69rg"
server_key: "6b06e1265b88f856ac7443f108d68a25"
server_key_iv: "4cf774346be6934574001de52da924a8"
server_key_version: "encryptionkey@0"
I20260806 21:09:35.050979 11816 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 21:09:35.053939 11831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:35.055377 11816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 21:09:35.055565 11816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "387028fb73dc4583ad3c8fd99446462d"
format_stamp: "Formatted at 2026-08-06 21:09:35 on dist-test-slave-69rg"
server_key: "6b06e1265b88f856ac7443f108d68a25"
server_key_iv: "4cf774346be6934574001de52da924a8"
server_key_version: "encryptionkey@0"
I20260806 21:09:35.055716 11816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:35.081610 11816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:35.086781 11816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:35.087054 11816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:35.095335 11816 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:41497
I20260806 21:09:35.095706 11884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:41497 every 8 connection(s)
I20260806 21:09:35.096472 11816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:09:35.099828 11885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:09:35.102627 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 11816
I20260806 21:09:35.102847 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:09:35.103188 28551 external_mini_cluster.cc:1428] Setting key 412ccb0c71a2d27c865e69db22fca00f
I20260806 21:09:35.106655 11885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d: Bootstrap starting.
I20260806 21:09:35.109048 11885 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d: Neither blocks nor log segments found. Creating new log.
I20260806 21:09:35.109942 11885 log.cc:824] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d: Log is configured to *not* fsync() on all Append() calls
I20260806 21:09:35.112660 11885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d: No bootstrap required, opened a new log
Aug 06 21:09:35 dist-test-slave-69rg krb5kdc[11803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:09:35.117781 11885 raft_consensus.cc:348] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387028fb73dc4583ad3c8fd99446462d" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41497 } }
I20260806 21:09:35.118176 11885 raft_consensus.cc:374] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:09:35.118281 11885 raft_consensus.cc:729] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 387028fb73dc4583ad3c8fd99446462d, State: Initialized, Role: FOLLOWER
I20260806 21:09:35.118847 11885 consensus_queue.cc:260] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387028fb73dc4583ad3c8fd99446462d" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41497 } }
I20260806 21:09:35.119035 11885 raft_consensus.cc:388] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:09:35.119124 11885 raft_consensus.cc:482] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:09:35.119226 11885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:09:35.119776 11888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:35.104736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56390 (local address 127.27.225.254:41497)
0806 21:09:35.105299 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:35.105318 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:35.105369 (+ 51us) server_negotiation.cc:407] Connection header received
0806 21:09:35.106185 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:35.106206 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:35.106556 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:35.106928 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:35.107819 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.108833 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.109790 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.110428 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.110655 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:35.113653 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:35.113660 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:35.113679 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:35.113713 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:35.116002 (+ 2289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:35.116582 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:35.116590 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:35.116596 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:35.116753 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:35.117048 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:35.117051 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:35.117053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:35.117645 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:35.117886 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:35.118246 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 21:09:35.118599 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":157,"threads_started":1}
I20260806 21:09:35.120577 11885 raft_consensus.cc:504] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387028fb73dc4583ad3c8fd99446462d" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41497 } }
I20260806 21:09:35.120981 11885 leader_election.cc:302] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 387028fb73dc4583ad3c8fd99446462d; no voters:
I20260806 21:09:35.121538 11885 leader_election.cc:288] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:09:35.121870 11890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:09:35.122114 11890 raft_consensus.cc:686] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [term 1 LEADER]: Becoming Leader. State: Replica: 387028fb73dc4583ad3c8fd99446462d, State: Running, Role: LEADER
I20260806 21:09:35.122514 11890 consensus_queue.cc:237] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387028fb73dc4583ad3c8fd99446462d" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41497 } }
I20260806 21:09:35.123245 11885 sys_catalog.cc:564] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:09:35.124472 11891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 387028fb73dc4583ad3c8fd99446462d. Latest consensus state: current_term: 1 leader_uuid: "387028fb73dc4583ad3c8fd99446462d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387028fb73dc4583ad3c8fd99446462d" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41497 } } }
I20260806 21:09:35.124639 11891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [sys.catalog]: This master's current role is: LEADER
I20260806 21:09:35.125061 11899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:09:35.125039 11892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "387028fb73dc4583ad3c8fd99446462d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387028fb73dc4583ad3c8fd99446462d" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41497 } } }
I20260806 21:09:35.125216 11892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [sys.catalog]: This master's current role is: LEADER
I20260806 21:09:35.128284 11899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:09:35.134639 11899 catalog_manager.cc:1347] Generated new cluster ID: 32aac675b8a74103af16b25e6c1eb8ed
I20260806 21:09:35.134814 11899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:09:35.148458 11899 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:09:35.149899 11899 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:09:35.155005 11899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d: Generated new TSK 0
I20260806 21:09:35.155818 11899 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.225.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
I20260806 21:09:35.198843 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:41497
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:40747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:09:35.317333 11911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:35.317639 11911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:35.317687 11911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:35.321903 11911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:35.321979 11911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:35.322041 11911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:09:35.327853 11911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:40747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:41497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:35.329008 11911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:35.330104 11911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:35.339980 11920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:35.339982 11917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:35.339982 11918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:35.340420 11911 server_base.cc:1034] running on GCE node
I20260806 21:09:35.340854 11911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:35.341504 11911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:35.342703 11911 hybrid_clock.cc:648] HybridClock initialized: now 1786050575342674 us; error 40 us; skew 500 ppm
Aug 06 21:09:35 dist-test-slave-69rg krb5kdc[11803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050575, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:35.345901 11911 init.cc:377] Logged in from keytab as kudu/127.27.225.193@KRBTEST.COM (short username kudu)
I20260806 21:09:35.346899 11911 webserver.cc:460] Webserver started at http://127.27.225.193:33775/ using document root <none> and password file <none>
I20260806 21:09:35.347506 11911 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:35.347606 11911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:35.347906 11911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:09:35.350164 11911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "05c8aa7600b14380a13ebc51085f91f2"
format_stamp: "Formatted at 2026-08-06 21:09:35 on dist-test-slave-69rg"
server_key: "eb205018129a7e4c425e97d29baa443f"
server_key_iv: "ce63ffda53a082266852ee1d07ba2ae7"
server_key_version: "encryptionkey@0"
I20260806 21:09:35.350761 11911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "05c8aa7600b14380a13ebc51085f91f2"
format_stamp: "Formatted at 2026-08-06 21:09:35 on dist-test-slave-69rg"
server_key: "eb205018129a7e4c425e97d29baa443f"
server_key_iv: "ce63ffda53a082266852ee1d07ba2ae7"
server_key_version: "encryptionkey@0"
I20260806 21:09:35.354763 11911 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 21:09:35.357555 11927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:35.358963 11911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 21:09:35.359110 11911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "05c8aa7600b14380a13ebc51085f91f2"
format_stamp: "Formatted at 2026-08-06 21:09:35 on dist-test-slave-69rg"
server_key: "eb205018129a7e4c425e97d29baa443f"
server_key_iv: "ce63ffda53a082266852ee1d07ba2ae7"
server_key_version: "encryptionkey@0"
I20260806 21:09:35.359237 11911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:35.378082 11911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:35.381270 11911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:35.381500 11911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:35.382182 11911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:09:35.383055 11911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:09:35.383131 11911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:35.383201 11911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:09:35.383241 11911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:35.392313 11911 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:32843
I20260806 21:09:35.392333 12041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:32843 every 8 connection(s)
I20260806 21:09:35.393225 11911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:09:35.397405 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 11911
I20260806 21:09:35.397575 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:09:35.397944 28551 external_mini_cluster.cc:1428] Setting key c10a7a3238b054666874bdf8b1806e15
Aug 06 21:09:35 dist-test-slave-69rg krb5kdc[11803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050575, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:09:35.408200 11888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:35.395282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:50891 (local address 127.27.225.254:41497)
0806 21:09:35.395427 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:35.395432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:35.396459 (+ 1027us) server_negotiation.cc:407] Connection header received
0806 21:09:35.397415 (+ 956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:35.397418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:35.397463 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:35.397586 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:35.399083 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.399766 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.400959 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.401194 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.401321 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:35.404171 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:35.404174 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:35.404177 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:35.404212 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:35.405763 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:35.406451 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:35.406456 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:35.406457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:35.406569 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:35.406989 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:35.406993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:35.406994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:35.407207 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:35.407324 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:35.407883 (+ 559us) server_negotiation.cc:299] Negotiation successful
0806 21:09:35.408025 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
WARNING: no policy specified for kudu/127.27.225.194@KRBTEST.COM; defaulting to no policy
I20260806 21:09:35.409145 12044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:35.395595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.193:50891)
0806 21:09:35.396314 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:35.396355 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:35.397197 (+ 842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:35.397780 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:35.397787 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:35.398361 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:35.398925 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:35.398939 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.399912 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.399915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:35.400816 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:35.400826 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.401375 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.401379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:35.401550 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:35.402212 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:35.402229 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:35.403971 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:35.405977 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:35.405984 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:35.406004 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:35.406313 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:35.406687 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:35.406691 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:35.406693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:35.406880 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:35.407331 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:35.407341 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:35.407626 (+ 285us) client_negotiation.cc:769] Sending connection context
0806 21:09:35.407833 (+ 207us) client_negotiation.cc:240] Negotiation successful
0806 21:09:35.408204 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":117,"threads_started":1}
I20260806 21:09:35.410674 12042 heartbeater.cc:347] Connected to a master server at 127.27.225.254:41497
I20260806 21:09:35.410987 12042 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:35.411571 12042 heartbeater.cc:511] Master 127.27.225.254:41497 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.194@KRBTEST.COM" created.
I20260806 21:09:35.413419 11846 ts_manager.cc:194] Registered new tserver with Master: 05c8aa7600b14380a13ebc51085f91f2 (127.27.225.193:32843)
I20260806 21:09:35.414732 11846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.193@KRBTEST.COM'} at 127.27.225.193:50891
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
I20260806 21:09:35.429430 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:41497
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:40747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:09:35.551105 12047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:35.551376 12047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:35.551419 12047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:35.555447 12047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:35.555522 12047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:35.555608 12047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:09:35.561012 12047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:40747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:41497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:35.562090 12047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:35.563129 12047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:35.572892 12053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:35.572988 12047 server_base.cc:1034] running on GCE node
W20260806 21:09:35.572880 12052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:35.572880 12055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:35.573522 12047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:35.574232 12047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:35.575389 12047 hybrid_clock.cc:648] HybridClock initialized: now 1786050575575374 us; error 28 us; skew 500 ppm
Aug 06 21:09:35 dist-test-slave-69rg krb5kdc[11803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050575, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:35.578440 12047 init.cc:377] Logged in from keytab as kudu/127.27.225.194@KRBTEST.COM (short username kudu)
I20260806 21:09:35.579450 12047 webserver.cc:460] Webserver started at http://127.27.225.194:36011/ using document root <none> and password file <none>
I20260806 21:09:35.580039 12047 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:35.580106 12047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:35.580332 12047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:09:35.582598 12047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "3000883cceea4af6a10cb3b59231f5d6"
format_stamp: "Formatted at 2026-08-06 21:09:35 on dist-test-slave-69rg"
server_key: "633da2a19380701e05ded2d340c14837"
server_key_iv: "d22e2c10032fec14e6853308a64d640b"
server_key_version: "encryptionkey@0"
I20260806 21:09:35.583213 12047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "3000883cceea4af6a10cb3b59231f5d6"
format_stamp: "Formatted at 2026-08-06 21:09:35 on dist-test-slave-69rg"
server_key: "633da2a19380701e05ded2d340c14837"
server_key_iv: "d22e2c10032fec14e6853308a64d640b"
server_key_version: "encryptionkey@0"
I20260806 21:09:35.587225 12047 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:09:35.589715 12062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:35.591004 12047 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:09:35.591137 12047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "3000883cceea4af6a10cb3b59231f5d6"
format_stamp: "Formatted at 2026-08-06 21:09:35 on dist-test-slave-69rg"
server_key: "633da2a19380701e05ded2d340c14837"
server_key_iv: "d22e2c10032fec14e6853308a64d640b"
server_key_version: "encryptionkey@0"
I20260806 21:09:35.591256 12047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:35.607916 12047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:35.611155 12047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:35.611363 12047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:35.611967 12047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:09:35.612797 12047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:09:35.612845 12047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:35.612911 12047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:09:35.612952 12047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:35.622279 12047 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:33975
I20260806 21:09:35.622303 12180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:33975 every 8 connection(s)
I20260806 21:09:35.623281 12047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:09:35.627468 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 12047
I20260806 21:09:35.627629 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:09:35.627983 28551 external_mini_cluster.cc:1428] Setting key 4917888bb9aa5a342ff4f8f96aeb621d
Aug 06 21:09:35 dist-test-slave-69rg krb5kdc[11803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050575, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.225.195@KRBTEST.COM; defaulting to no policy
I20260806 21:09:35.639149 11888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:35.625376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:47599 (local address 127.27.225.254:41497)
0806 21:09:35.625566 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:35.625570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:35.626713 (+ 1143us) server_negotiation.cc:407] Connection header received
0806 21:09:35.627810 (+ 1097us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:35.627813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:35.627862 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:35.627940 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:35.629420 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.630224 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.631444 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.631851 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.631952 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:35.635109 (+ 3157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:35.635118 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:35.635120 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:35.635150 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:35.636867 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:35.637445 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:35.637449 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:35.637451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:35.637540 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:35.637934 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:35.637939 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:35.637940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:35.638169 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:35.638282 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:35.638829 (+ 547us) server_negotiation.cc:299] Negotiation successful
0806 21:09:35.638964 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260806 21:09:35.639983 12183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:35.625880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.194:47599)
0806 21:09:35.626561 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:35.626602 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:35.627568 (+ 966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:35.628125 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:35.628134 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:35.628599 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:35.629240 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:35.629262 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.630362 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.630365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:35.631307 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:35.631318 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.632043 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.632048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:35.632322 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:35.633043 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:35.633071 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:35.634909 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:35.637016 (+ 2107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:35.637023 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:35.637045 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:35.637334 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:35.637647 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:35.637650 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:35.637652 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:35.637805 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:35.638280 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:35.638287 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:35.638568 (+ 281us) client_negotiation.cc:769] Sending connection context
0806 21:09:35.638770 (+ 202us) client_negotiation.cc:240] Negotiation successful
0806 21:09:35.639117 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":106,"threads_started":1}
I20260806 21:09:35.641530 12181 heartbeater.cc:347] Connected to a master server at 127.27.225.254:41497
I20260806 21:09:35.641793 12181 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.195@KRBTEST.COM" created.
I20260806 21:09:35.642561 12181 heartbeater.cc:511] Master 127.27.225.254:41497 requested a full tablet report, sending...
I20260806 21:09:35.643734 11846 ts_manager.cc:194] Registered new tserver with Master: 3000883cceea4af6a10cb3b59231f5d6 (127.27.225.194:33975)
I20260806 21:09:35.644600 11846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.194@KRBTEST.COM'} at 127.27.225.194:47599
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
I20260806 21:09:35.662220 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:41497
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:40747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:09:35.778905 12186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:35.779181 12186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:35.779322 12186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:35.783517 12186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:35.783591 12186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:35.783653 12186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:09:35.789136 12186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:40747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:41497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:35.790305 12186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:35.791361 12186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:35.801266 12194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:35.801384 12186 server_base.cc:1034] running on GCE node
W20260806 21:09:35.801272 12191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:35.801256 12192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:35.801995 12186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:35.802659 12186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:35.803848 12186 hybrid_clock.cc:648] HybridClock initialized: now 1786050575803819 us; error 41 us; skew 500 ppm
Aug 06 21:09:35 dist-test-slave-69rg krb5kdc[11803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050575, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:35.806788 12186 init.cc:377] Logged in from keytab as kudu/127.27.225.195@KRBTEST.COM (short username kudu)
I20260806 21:09:35.807749 12186 webserver.cc:460] Webserver started at http://127.27.225.195:38769/ using document root <none> and password file <none>
I20260806 21:09:35.808277 12186 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:35.808347 12186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:35.808523 12186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:09:35.810652 12186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "5e1256e1d36549b7a872dc6b51e79166"
format_stamp: "Formatted at 2026-08-06 21:09:35 on dist-test-slave-69rg"
server_key: "859e13c0f4faf322b761dd9797334d2f"
server_key_iv: "5f2658a7d0ba22b3adb28778b8098451"
server_key_version: "encryptionkey@0"
I20260806 21:09:35.811187 12186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "5e1256e1d36549b7a872dc6b51e79166"
format_stamp: "Formatted at 2026-08-06 21:09:35 on dist-test-slave-69rg"
server_key: "859e13c0f4faf322b761dd9797334d2f"
server_key_iv: "5f2658a7d0ba22b3adb28778b8098451"
server_key_version: "encryptionkey@0"
I20260806 21:09:35.814985 12186 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260806 21:09:35.817412 12201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:35.818763 12186 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:09:35.818897 12186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "5e1256e1d36549b7a872dc6b51e79166"
format_stamp: "Formatted at 2026-08-06 21:09:35 on dist-test-slave-69rg"
server_key: "859e13c0f4faf322b761dd9797334d2f"
server_key_iv: "5f2658a7d0ba22b3adb28778b8098451"
server_key_version: "encryptionkey@0"
I20260806 21:09:35.819022 12186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:35.829681 12186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:35.833213 12186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:35.833451 12186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:35.834156 12186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:09:35.835037 12186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:09:35.835110 12186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:35.835181 12186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:09:35.835217 12186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:35.844870 12186 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:40917
I20260806 21:09:35.844895 12316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:40917 every 8 connection(s)
I20260806 21:09:35.845789 12186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:09:35.850085 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 12186
I20260806 21:09:35.850234 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:09:35.850533 28551 external_mini_cluster.cc:1428] Setting key afb439eaded0d9089d4bf7bdbd196705
Aug 06 21:09:35 dist-test-slave-69rg krb5kdc[11803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050575, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:09:35.859351 11888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:35.847707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:58681 (local address 127.27.225.254:41497)
0806 21:09:35.847855 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:35.847859 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:35.848895 (+ 1036us) server_negotiation.cc:407] Connection header received
0806 21:09:35.849869 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:35.849873 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:35.849931 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:35.850028 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:35.851404 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.851999 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.853135 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.853374 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.853489 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:35.855955 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:35.855957 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:35.855960 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:35.855983 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:35.857244 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:35.857747 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:35.857751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:35.857752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:35.857915 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:35.858280 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:35.858283 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:35.858284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:35.858482 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:35.858572 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:35.859114 (+ 542us) server_negotiation.cc:299] Negotiation successful
0806 21:09:35.859227 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 21:09:35.860169 12319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:35.848060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.195:58681)
0806 21:09:35.848721 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:35.848802 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:35.849622 (+ 820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:35.850197 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:35.850225 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:35.850684 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:35.851265 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:35.851279 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.852133 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.852136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:35.853008 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:35.853017 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.853529 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.853533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:35.853711 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:35.854340 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:35.854356 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:35.855795 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:35.857360 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:35.857365 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:35.857376 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:35.857628 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:35.858010 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:35.858012 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:35.858014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:35.858161 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:35.858583 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:35.858588 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:35.858817 (+ 229us) client_negotiation.cc:769] Sending connection context
0806 21:09:35.859028 (+ 211us) client_negotiation.cc:240] Negotiation successful
0806 21:09:35.859290 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":115,"threads_started":1}
I20260806 21:09:35.861409 12317 heartbeater.cc:347] Connected to a master server at 127.27.225.254:41497
I20260806 21:09:35.861625 12317 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:35.862186 12317 heartbeater.cc:511] Master 127.27.225.254:41497 requested a full tablet report, sending...
I20260806 21:09:35.863445 11846 ts_manager.cc:194] Registered new tserver with Master: 5e1256e1d36549b7a872dc6b51e79166 (127.27.225.195:40917)
I20260806 21:09:35.864224 11846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.195@KRBTEST.COM'} at 127.27.225.195:58681
I20260806 21:09:35.864476 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:09:35.874256 11888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:35.866637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56394 (local address 127.27.225.254:41497)
0806 21:09:35.866796 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:35.866801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:35.866984 (+ 183us) server_negotiation.cc:407] Connection header received
0806 21:09:35.867141 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:35.867145 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:35.867195 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:35.867275 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:35.867984 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.868502 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.869336 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.869570 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.869683 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:35.871113 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:35.871117 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:35.871119 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:35.871145 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:35.872444 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:35.872909 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:35.872912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:35.872914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:35.873005 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:35.873286 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:35.873291 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:35.873293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:35.873536 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:35.873682 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:35.873973 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 21:09:35.874095 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 21:09:35.877152 11846 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:09:35.880331 11846 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:09:35.895612 12334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:35.889927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55450 (local address 127.27.225.195:40917)
0806 21:09:35.890476 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:35.890489 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:35.890511 (+ 22us) server_negotiation.cc:407] Connection header received
0806 21:09:35.890584 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:35.890589 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:35.890773 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:35.891189 (+ 416us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:09:35.891380 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.892135 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.893842 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.894852 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.894982 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:35.895119 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:35.895305 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 21:09:35.895422 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":121,"threads_started":1}
I20260806 21:09:35.895675 12333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:35.889744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:40917 (local address 127.0.0.1:55450)
0806 21:09:35.890101 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:35.890116 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:35.890212 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:35.890933 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:35.890936 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:35.890964 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:09:35.891267 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:35.891277 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.892274 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.892281 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:35.893668 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:35.893679 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.894991 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.894995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:35.895101 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:35.895136 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 21:09:35.895245 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 21:09:35.895430 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":92,"spinlock_wait_cycles":768,"thread_start_us":127,"threads_started":1}
I20260806 21:09:35.897236 12332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:35.889477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45818 (local address 127.27.225.193:32843)
0806 21:09:35.889774 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:35.889789 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:35.890253 (+ 464us) server_negotiation.cc:407] Connection header received
0806 21:09:35.890355 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:35.890359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:35.890514 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:35.890655 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:09:35.891368 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.892581 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.895342 (+ 2761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.896410 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.896538 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:35.896650 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:35.896998 (+ 348us) server_negotiation.cc:299] Negotiation successful
0806 21:09:35.897104 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":58,"threads_started":1}
I20260806 21:09:35.897871 12330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:35.889098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:32843 (local address 127.0.0.1:45818)
0806 21:09:35.889954 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:35.889968 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:35.890111 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:35.890846 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:35.890852 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:35.890886 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:09:35.891213 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:35.891221 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.893991 (+ 2770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.893994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:35.895168 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:35.895178 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.896539 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.896543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:35.896699 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:35.896723 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 21:09:35.896809 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 21:09:35.897715 (+ 906us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":777,"mutex_wait_us":130,"spinlock_wait_cycles":15872,"thread_start_us":105,"threads_started":1}
I20260806 21:09:35.898034 12329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:35.889013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:33975 (local address 127.0.0.1:32776)
0806 21:09:35.889793 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:35.889890 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:35.890111 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:35.890858 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:35.890860 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:35.890885 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:09:35.891216 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:35.891221 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.892732 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.892736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:35.893877 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:35.893885 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.896539 (+ 2654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.896543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:35.896855 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:35.897699 (+ 844us) client_negotiation.cc:769] Sending connection context
0806 21:09:35.897788 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 21:09:35.897892 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"mutex_wait_us":1082,"spinlock_wait_cycles":7040,"thread_start_us":87,"threads_started":1}
I20260806 21:09:35.898152 12331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:35.889136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32776 (local address 127.27.225.194:33975)
0806 21:09:35.889574 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:35.889588 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:35.889987 (+ 399us) server_negotiation.cc:407] Connection header received
0806 21:09:35.890298 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:35.890303 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:35.890515 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:35.890648 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:09:35.891360 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.892574 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.895361 (+ 2787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:35.896254 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:35.896334 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:35.896420 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:35.897873 (+ 1453us) server_negotiation.cc:299] Negotiation successful
0806 21:09:35.897975 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":117,"threads_started":1}
I20260806 21:09:35.899350 12251 tablet_service.cc:1467] Processing CreateTablet for tablet fab4963c452142449f14603b62cc1222 (DEFAULT_TABLE table=test-table [id=2a7d7f54fd5d493a82f9f7ff2ba22782]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:09:35.900554 12251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fab4963c452142449f14603b62cc1222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:09:35.900554 11975 tablet_service.cc:1467] Processing CreateTablet for tablet fab4963c452142449f14603b62cc1222 (DEFAULT_TABLE table=test-table [id=2a7d7f54fd5d493a82f9f7ff2ba22782]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:09:35.901057 12114 tablet_service.cc:1467] Processing CreateTablet for tablet fab4963c452142449f14603b62cc1222 (DEFAULT_TABLE table=test-table [id=2a7d7f54fd5d493a82f9f7ff2ba22782]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:09:35.901471 11975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fab4963c452142449f14603b62cc1222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:09:35.901994 12114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fab4963c452142449f14603b62cc1222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:09:35.907020 12335 tablet_bootstrap.cc:492] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2: Bootstrap starting.
I20260806 21:09:35.908456 12336 tablet_bootstrap.cc:492] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166: Bootstrap starting.
I20260806 21:09:35.908591 12337 tablet_bootstrap.cc:492] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6: Bootstrap starting.
I20260806 21:09:35.909389 12335 tablet_bootstrap.cc:654] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2: Neither blocks nor log segments found. Creating new log.
I20260806 21:09:35.910548 12336 tablet_bootstrap.cc:654] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166: Neither blocks nor log segments found. Creating new log.
I20260806 21:09:35.910778 12335 log.cc:824] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2: Log is configured to *not* fsync() on all Append() calls
I20260806 21:09:35.911237 12337 tablet_bootstrap.cc:654] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6: Neither blocks nor log segments found. Creating new log.
I20260806 21:09:35.911995 12336 log.cc:824] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166: Log is configured to *not* fsync() on all Append() calls
I20260806 21:09:35.912050 12337 log.cc:824] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6: Log is configured to *not* fsync() on all Append() calls
I20260806 21:09:35.913210 12335 tablet_bootstrap.cc:492] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2: No bootstrap required, opened a new log
I20260806 21:09:35.913444 12335 ts_tablet_manager.cc:1397] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 21:09:35.913980 12337 tablet_bootstrap.cc:492] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6: No bootstrap required, opened a new log
I20260806 21:09:35.913985 12336 tablet_bootstrap.cc:492] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166: No bootstrap required, opened a new log
I20260806 21:09:35.914163 12336 ts_tablet_manager.cc:1397] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 21:09:35.914170 12337 ts_tablet_manager.cc:1397] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 21:09:35.916492 12335 raft_consensus.cc:348] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } } peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } }
I20260806 21:09:35.916817 12335 raft_consensus.cc:374] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:09:35.916895 12335 raft_consensus.cc:729] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05c8aa7600b14380a13ebc51085f91f2, State: Initialized, Role: FOLLOWER
I20260806 21:09:35.917419 12335 consensus_queue.cc:260] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } } peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } }
I20260806 21:09:35.918174 12042 heartbeater.cc:503] Master 127.27.225.254:41497 was elected leader, sending a full tablet report...
I20260806 21:09:35.917981 12336 raft_consensus.cc:348] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } } peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } }
I20260806 21:09:35.918256 12336 raft_consensus.cc:374] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:09:35.918354 12336 raft_consensus.cc:729] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e1256e1d36549b7a872dc6b51e79166, State: Initialized, Role: FOLLOWER
I20260806 21:09:35.918542 12335 ts_tablet_manager.cc:1428] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 21:09:35.918843 12336 consensus_queue.cc:260] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } } peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } }
I20260806 21:09:35.917981 12337 raft_consensus.cc:348] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } } peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } }
I20260806 21:09:35.919447 12337 raft_consensus.cc:374] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:09:35.919528 12317 heartbeater.cc:503] Master 127.27.225.254:41497 was elected leader, sending a full tablet report...
I20260806 21:09:35.919538 12337 raft_consensus.cc:729] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3000883cceea4af6a10cb3b59231f5d6, State: Initialized, Role: FOLLOWER
I20260806 21:09:35.919801 12336 ts_tablet_manager.cc:1428] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 21:09:35.919970 12337 consensus_queue.cc:260] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } } peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } }
I20260806 21:09:35.921041 12337 ts_tablet_manager.cc:1428] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260806 21:09:35.921389 12181 heartbeater.cc:503] Master 127.27.225.254:41497 was elected leader, sending a full tablet report...
W20260806 21:09:36.096858 12318 tablet.cc:2369] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 21:09:36.124768 12182 tablet.cc:2369] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 21:09:36.144637 12043 tablet.cc:2369] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:09:36.218006 12343 raft_consensus.cc:482] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:09:36.218278 12343 raft_consensus.cc:504] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } } peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } }
I20260806 21:09:36.219622 12343 leader_election.cc:288] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3000883cceea4af6a10cb3b59231f5d6 (127.27.225.194:33975), 05c8aa7600b14380a13ebc51085f91f2 (127.27.225.193:32843)
I20260806 21:09:36.220082 12344 raft_consensus.cc:482] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:09:36.220289 12344 raft_consensus.cc:504] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } } peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } }
I20260806 21:09:36.221657 12344 leader_election.cc:288] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05c8aa7600b14380a13ebc51085f91f2 (127.27.225.193:32843), 5e1256e1d36549b7a872dc6b51e79166 (127.27.225.195:40917)
I20260806 21:09:36.224573 12319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:36.219859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:33975 (local address 127.27.225.195:53667)
0806 21:09:36.219971 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:36.219988 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:36.220095 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:36.220762 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:36.220765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:36.220782 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:09:36.220960 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:36.220969 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:36.221864 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:36.221867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:36.223124 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:36.223134 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:36.224141 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:36.224144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:36.224252 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:36.224276 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 21:09:36.224361 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 21:09:36.224430 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":26}
I20260806 21:09:36.225069 12331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:36.220372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:53667 (local address 127.27.225.194:33975)
0806 21:09:36.220483 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:36.220488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:36.220509 (+ 21us) server_negotiation.cc:407] Connection header received
0806 21:09:36.220558 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:36.220560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:36.220633 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:36.220724 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:09:36.221107 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:36.221711 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:36.223296 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:36.223879 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:36.224006 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:36.224068 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:36.224859 (+ 791us) server_negotiation.cc:299] Negotiation successful
0806 21:09:36.224949 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 21:09:36.225513 12332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:36.220177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:44917 (local address 127.27.225.193:32843)
0806 21:09:36.220308 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:36.220313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:36.220470 (+ 157us) server_negotiation.cc:407] Connection header received
0806 21:09:36.220684 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:36.220688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:36.220744 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:36.220817 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:09:36.221278 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:36.222011 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:36.224154 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:36.224743 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:36.224834 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:36.224881 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:36.225305 (+ 424us) server_negotiation.cc:299] Negotiation successful
0806 21:09:36.225386 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 21:09:36.225695 12134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fab4963c452142449f14603b62cc1222" candidate_uuid: "5e1256e1d36549b7a872dc6b51e79166" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3000883cceea4af6a10cb3b59231f5d6" is_pre_election: true
I20260806 21:09:36.225950 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:36.219968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:32843 (local address 127.27.225.195:44917)
0806 21:09:36.220377 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:36.220391 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:36.220539 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:36.220872 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:36.220876 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:36.220897 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:09:36.221095 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:36.221102 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:36.222664 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:36.222667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:36.223972 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:36.223981 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:36.225023 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:36.225028 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:36.225131 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:36.225172 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 21:09:36.225705 (+ 533us) client_negotiation.cc:240] Negotiation successful
0806 21:09:36.225785 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":29,"thread_start_us":164,"threads_started":1}
I20260806 21:09:36.225962 12134 raft_consensus.cc:2454] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e1256e1d36549b7a872dc6b51e79166 in term 0.
I20260806 21:09:36.226485 12203 leader_election.cc:302] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3000883cceea4af6a10cb3b59231f5d6, 5e1256e1d36549b7a872dc6b51e79166; no voters:
I20260806 21:09:36.226558 11995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fab4963c452142449f14603b62cc1222" candidate_uuid: "5e1256e1d36549b7a872dc6b51e79166" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05c8aa7600b14380a13ebc51085f91f2" is_pre_election: true
I20260806 21:09:36.226831 12343 raft_consensus.cc:2781] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:09:36.226884 11995 raft_consensus.cc:2454] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e1256e1d36549b7a872dc6b51e79166 in term 0.
I20260806 21:09:36.226934 12343 raft_consensus.cc:482] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:09:36.226985 12343 raft_consensus.cc:3037] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:09:36.227094 12348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:36.222224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:40917 (local address 127.27.225.194:50653)
0806 21:09:36.222537 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:36.222548 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:36.222623 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:36.222971 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:36.222975 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:36.222990 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:09:36.223099 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:36.223105 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:36.224094 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:36.224100 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:36.225394 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:36.225406 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:36.226425 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:36.226429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:36.226520 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:36.226541 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 21:09:36.226862 (+ 321us) client_negotiation.cc:240] Negotiation successful
0806 21:09:36.226941 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":82,"threads_started":1}
I20260806 21:09:36.226840 12334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:36.222510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:50653 (local address 127.27.225.195:40917)
0806 21:09:36.222599 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:36.222603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:36.222615 (+ 12us) server_negotiation.cc:407] Connection header received
0806 21:09:36.222743 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:36.222747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:36.222787 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:36.222835 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:09:36.223216 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:36.223955 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:36.225531 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:36.226184 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:36.226271 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:36.226329 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:36.226656 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 21:09:36.226732 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 21:09:36.227914 12346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:36.221861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:32843 (local address 127.27.225.194:43377)
0806 21:09:36.222241 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:36.222255 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:36.222357 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:36.222725 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:36.222729 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:36.222743 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:09:36.222923 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:36.222933 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:36.224180 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:36.224183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:36.225892 (+ 1709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:36.225904 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:36.227466 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:36.227469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:36.227573 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:36.227598 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 21:09:36.227681 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 21:09:36.227761 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":91,"threads_started":1}
I20260806 21:09:36.227926 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:36.221983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:43377 (local address 127.27.225.193:32843)
0806 21:09:36.222481 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:36.222490 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:36.222503 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:09:36.222543 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:36.222548 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:36.222602 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:36.223082 (+ 480us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:09:36.223195 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:36.224067 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:36.226200 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:36.227332 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:36.227474 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:36.227521 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:36.227723 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 21:09:36.227797 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":131,"threads_started":1}
I20260806 21:09:36.228305 12271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fab4963c452142449f14603b62cc1222" candidate_uuid: "3000883cceea4af6a10cb3b59231f5d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e1256e1d36549b7a872dc6b51e79166" is_pre_election: true
I20260806 21:09:36.228399 12343 raft_consensus.cc:504] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } } peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } }
I20260806 21:09:36.228603 12271 raft_consensus.cc:2379] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3000883cceea4af6a10cb3b59231f5d6 in current term 1: Already voted for candidate 5e1256e1d36549b7a872dc6b51e79166 in this term.
I20260806 21:09:36.228770 12343 leader_election.cc:288] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166 [CANDIDATE]: Term 1 election: Requested vote from peers 3000883cceea4af6a10cb3b59231f5d6 (127.27.225.194:33975), 05c8aa7600b14380a13ebc51085f91f2 (127.27.225.193:32843)
I20260806 21:09:36.228766 11995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fab4963c452142449f14603b62cc1222" candidate_uuid: "3000883cceea4af6a10cb3b59231f5d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05c8aa7600b14380a13ebc51085f91f2" is_pre_election: true
I20260806 21:09:36.228930 11995 raft_consensus.cc:2454] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3000883cceea4af6a10cb3b59231f5d6 in term 0.
I20260806 21:09:36.229101 12134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fab4963c452142449f14603b62cc1222" candidate_uuid: "5e1256e1d36549b7a872dc6b51e79166" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3000883cceea4af6a10cb3b59231f5d6"
I20260806 21:09:36.229225 11994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fab4963c452142449f14603b62cc1222" candidate_uuid: "5e1256e1d36549b7a872dc6b51e79166" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05c8aa7600b14380a13ebc51085f91f2"
I20260806 21:09:36.229254 12134 raft_consensus.cc:3037] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:09:36.229339 11994 raft_consensus.cc:3037] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:09:36.229344 12066 leader_election.cc:302] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05c8aa7600b14380a13ebc51085f91f2, 3000883cceea4af6a10cb3b59231f5d6; no voters: 5e1256e1d36549b7a872dc6b51e79166
I20260806 21:09:36.230556 12134 raft_consensus.cc:2454] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e1256e1d36549b7a872dc6b51e79166 in term 1.
I20260806 21:09:36.230556 11994 raft_consensus.cc:2454] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e1256e1d36549b7a872dc6b51e79166 in term 1.
I20260806 21:09:36.230692 12344 raft_consensus.cc:2741] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260806 21:09:36.230935 12203 leader_election.cc:302] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 05c8aa7600b14380a13ebc51085f91f2, 3000883cceea4af6a10cb3b59231f5d6, 5e1256e1d36549b7a872dc6b51e79166; no voters:
I20260806 21:09:36.231104 12343 raft_consensus.cc:2781] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:09:36.231338 12343 raft_consensus.cc:686] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166 [term 1 LEADER]: Becoming Leader. State: Replica: 5e1256e1d36549b7a872dc6b51e79166, State: Running, Role: LEADER
I20260806 21:09:36.231668 12343 consensus_queue.cc:237] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } } peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } }
I20260806 21:09:36.235044 11848 catalog_manager.cc:5495] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e1256e1d36549b7a872dc6b51e79166 (127.27.225.195). New cstate: current_term: 1 leader_uuid: "5e1256e1d36549b7a872dc6b51e79166" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } health_report { overall_health: HEALTHY } } }
I20260806 21:09:36.267262 12334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:36.263225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55454 (local address 127.27.225.195:40917)
0806 21:09:36.263369 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:36.263373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:36.263407 (+ 34us) server_negotiation.cc:407] Connection header received
0806 21:09:36.263527 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:36.263529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:36.263589 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:36.263694 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:09:36.263918 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:36.264549 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:36.265479 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:36.265784 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:36.265951 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:36.266377 (+ 426us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:09:36.266826 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:09:36.266936 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:36.266993 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 21:09:36.267049 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 21:09:36.273439 11994 raft_consensus.cc:1261] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2 [term 1 FOLLOWER]: Refusing update from remote peer 5e1256e1d36549b7a872dc6b51e79166: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:09:36.273439 12134 raft_consensus.cc:1261] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6 [term 1 FOLLOWER]: Refusing update from remote peer 5e1256e1d36549b7a872dc6b51e79166: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:09:36.274154 12349 consensus_queue.cc:1035] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:09:36.274377 12343 consensus_queue.cc:1035] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:09:36.282502 12353 mvcc.cc:205] Tried to move back new op lower bound from 7315663160407576576 to 7315663160247427072. Current Snapshot: MvccSnapshot[applied={T|T < 7315663160407576576}]
I20260806 21:09:36.283834 12354 mvcc.cc:205] Tried to move back new op lower bound from 7315663160407576576 to 7315663160247427072. Current Snapshot: MvccSnapshot[applied={T|T < 7315663160407576576}]
I20260806 21:09:36.292400 11847 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:09:36.292611 11847 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:09:36.295245 11847 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d: Sending DeleteTablet for 3 replicas of tablet fab4963c452142449f14603b62cc1222
I20260806 21:09:36.296172 12251 tablet_service.cc:1514] Processing DeleteTablet for tablet fab4963c452142449f14603b62cc1222 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:09:36 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:55450
I20260806 21:09:36.296195 11975 tablet_service.cc:1514] Processing DeleteTablet for tablet fab4963c452142449f14603b62cc1222 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:09:36 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:45818
I20260806 21:09:36.296453 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 11911
I20260806 21:09:36.297289 12114 tablet_service.cc:1514] Processing DeleteTablet for tablet fab4963c452142449f14603b62cc1222 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:09:36 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:32776
I20260806 21:09:36.298169 12365 tablet_replica.cc:330] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6: stopping tablet replica
I20260806 21:09:36.298630 12365 raft_consensus.cc:2229] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:09:36.298699 12364 tablet_replica.cc:330] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166: stopping tablet replica
I20260806 21:09:36.299023 12364 raft_consensus.cc:2229] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:09:36.299214 12365 raft_consensus.cc:2258] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:09:36.299466 12364 raft_consensus.cc:2258] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:09:36.300989 12364 ts_tablet_manager.cc:1905] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:09:36.303720 12365 ts_tablet_manager.cc:1905] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:09:36.304790 12364 ts_tablet_manager.cc:1918] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:09:36.304903 12364 log.cc:1196] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/fab4963c452142449f14603b62cc1222
I20260806 21:09:36.305289 12364 ts_tablet_manager.cc:1939] T fab4963c452142449f14603b62cc1222 P 5e1256e1d36549b7a872dc6b51e79166: Deleting consensus metadata
W20260806 21:09:36.306128 11834 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.225.193:32843 (error 108)
I20260806 21:09:36.306753 11834 catalog_manager.cc:4841] TS 5e1256e1d36549b7a872dc6b51e79166 (127.27.225.195:40917): tablet fab4963c452142449f14603b62cc1222 (table test-table [id=2a7d7f54fd5d493a82f9f7ff2ba22782]) successfully deleted
I20260806 21:09:36.307173 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 12047
I20260806 21:09:36.310489 12330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:36.310037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:32843 (local address 127.0.0.1:45828)
0806 21:09:36.310285 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:36.310372 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.193:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260806 21:09:36.310693 11834 catalog_manager.cc:4584] TS 05c8aa7600b14380a13ebc51085f91f2 (127.27.225.193:32843): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fab4963c452142449f14603b62cc1222: Network error: Client connection negotiation failed: client connection to 127.27.225.193:32843: connect: Connection refused (error 111)
I20260806 21:09:36.315322 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 12186
I20260806 21:09:36.316349 12330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:36.315950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:33975 (local address 127.0.0.1:32782)
0806 21:09:36.316168 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:36.316234 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.194:33975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260806 21:09:36.322157 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 11816
I20260806 21:09:56.330309 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:41497
--webserver_interface=127.27.225.254
--webserver_port=38293
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:40747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:41497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:09:56.447175 12367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:56.447456 12367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:56.447499 12367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:56.451679 12367 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 21:09:56.451748 12367 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 21:09:56.451766 12367 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 21:09:56.451779 12367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:09:56.451797 12367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:56.451817 12367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:56.451833 12367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:09:56.451889 12367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:09:56.457322 12367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:40747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:41497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:41497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=38293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:56.458583 12367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:56.459645 12367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:09:56.474653 12367 server_base.cc:1034] running on GCE node
W20260806 21:09:56.474598 12373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:56.474598 12374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:56.474610 12376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:56.475533 12367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:56.476835 12367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:56.478045 12367 hybrid_clock.cc:648] HybridClock initialized: now 1786050596478024 us; error 34 us; skew 500 ppm
Aug 06 21:09:56 dist-test-slave-69rg krb5kdc[11803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050596, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:56.485502 12367 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:09:56.486770 12367 webserver.cc:460] Webserver started at http://127.27.225.254:38293/ using document root <none> and password file <none>
I20260806 21:09:56.487442 12367 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:56.487586 12367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:56.492050 12367 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260806 21:09:56.494447 12383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:56.495757 12367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:09:56.495919 12367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "387028fb73dc4583ad3c8fd99446462d"
format_stamp: "Formatted at 2026-08-06 21:09:35 on dist-test-slave-69rg"
server_key: "6b06e1265b88f856ac7443f108d68a25"
server_key_iv: "4cf774346be6934574001de52da924a8"
server_key_version: "encryptionkey@0"
I20260806 21:09:56.496584 12367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:56.509156 12367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:56.512970 12367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:56.513193 12367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:56.521332 12367 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:41497
I20260806 21:09:56.521680 12436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:41497 every 8 connection(s)
I20260806 21:09:56.522678 12367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:09:56.527849 12437 sys_catalog.cc:263] Verifying existing consensus state
I20260806 21:09:56.528734 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 12367
I20260806 21:09:56.529284 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:32843
--local_ip_for_outbound_sockets=127.27.225.193
--tserver_master_addrs=127.27.225.254:41497
--webserver_port=33775
--webserver_interface=127.27.225.193
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:40747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:09:56.530525 12437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d: Bootstrap starting.
I20260806 21:09:56.550458 12437 log.cc:824] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d: Log is configured to *not* fsync() on all Append() calls
I20260806 21:09:56.560369 12437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 21:09:56.560850 12437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d: Bootstrap complete.
I20260806 21:09:56.565725 12437 raft_consensus.cc:348] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387028fb73dc4583ad3c8fd99446462d" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41497 } }
I20260806 21:09:56.566250 12437 raft_consensus.cc:729] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 387028fb73dc4583ad3c8fd99446462d, State: Initialized, Role: FOLLOWER
I20260806 21:09:56.566874 12437 consensus_queue.cc:260] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387028fb73dc4583ad3c8fd99446462d" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41497 } }
I20260806 21:09:56.567004 12437 raft_consensus.cc:388] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:09:56.567116 12437 raft_consensus.cc:482] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:09:56.567243 12437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [term 1 FOLLOWER]: Advancing to term 2
I20260806 21:09:56.569396 12437 raft_consensus.cc:504] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387028fb73dc4583ad3c8fd99446462d" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41497 } }
I20260806 21:09:56.569773 12437 leader_election.cc:302] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 387028fb73dc4583ad3c8fd99446462d; no voters:
I20260806 21:09:56.570273 12437 leader_election.cc:288] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 21:09:56.570370 12442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [term 2 FOLLOWER]: Leader election won for term 2
I20260806 21:09:56.570667 12442 raft_consensus.cc:686] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [term 2 LEADER]: Becoming Leader. State: Replica: 387028fb73dc4583ad3c8fd99446462d, State: Running, Role: LEADER
I20260806 21:09:56.571054 12442 consensus_queue.cc:237] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387028fb73dc4583ad3c8fd99446462d" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41497 } }
I20260806 21:09:56.571714 12437 sys_catalog.cc:564] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:09:56.572662 12444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 387028fb73dc4583ad3c8fd99446462d. Latest consensus state: current_term: 2 leader_uuid: "387028fb73dc4583ad3c8fd99446462d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387028fb73dc4583ad3c8fd99446462d" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41497 } } }
I20260806 21:09:56.572821 12444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [sys.catalog]: This master's current role is: LEADER
I20260806 21:09:56.573112 12443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "387028fb73dc4583ad3c8fd99446462d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387028fb73dc4583ad3c8fd99446462d" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41497 } } }
I20260806 21:09:56.573191 12443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [sys.catalog]: This master's current role is: LEADER
I20260806 21:09:56.574792 12451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:09:56.579916 12451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:09:56.580586 12451 catalog_manager.cc:1259] Loaded cluster ID: 32aac675b8a74103af16b25e6c1eb8ed
I20260806 21:09:56.580653 12451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:09:56.583106 12451 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:09:56.583818 12451 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d: Loaded TSK: 0
I20260806 21:09:56.617461 12451 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d: Generated new TSK 1
I20260806 21:09:56.617787 12451 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 21:09:56.657298 12439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:56.657650 12439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:56.657737 12439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:56.661960 12439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:56.662067 12439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:56.662161 12439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:09:56.667936 12439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:40747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:32843
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=33775
--tserver_master_addrs=127.27.225.254:41497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:56.669109 12439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:56.670375 12439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:56.681849 12465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:56.681872 12469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:56.681874 12467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:56.682472 12439 server_base.cc:1034] running on GCE node
I20260806 21:09:56.682951 12439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:56.683763 12439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:56.684983 12439 hybrid_clock.cc:648] HybridClock initialized: now 1786050596684944 us; error 47 us; skew 500 ppm
Aug 06 21:09:56 dist-test-slave-69rg krb5kdc[11803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050596, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:56.688479 12439 init.cc:377] Logged in from keytab as kudu/127.27.225.193@KRBTEST.COM (short username kudu)
I20260806 21:09:56.689630 12439 webserver.cc:460] Webserver started at http://127.27.225.193:33775/ using document root <none> and password file <none>
I20260806 21:09:56.690351 12439 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:56.690459 12439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:56.695655 12439 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 21:09:56.698413 12478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:56.700096 12439 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 21:09:56.700233 12439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "05c8aa7600b14380a13ebc51085f91f2"
format_stamp: "Formatted at 2026-08-06 21:09:35 on dist-test-slave-69rg"
server_key: "eb205018129a7e4c425e97d29baa443f"
server_key_iv: "ce63ffda53a082266852ee1d07ba2ae7"
server_key_version: "encryptionkey@0"
I20260806 21:09:56.700996 12439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:56.729537 12439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:56.734620 12439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:56.734838 12439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:56.735582 12439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:09:56.737022 12485 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 21:09:56.741369 12439 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 21:09:56.741472 12439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260806 21:09:56.741545 12439 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 21:09:56.744016 12439 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 21:09:56.744086 12439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.001s sys 0.000s
I20260806 21:09:56.744439 12485 tablet_bootstrap.cc:492] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2: Bootstrap starting.
I20260806 21:09:56.754379 12439 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:32843
I20260806 21:09:56.754603 12594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:32843 every 8 connection(s)
I20260806 21:09:56.755524 12439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:09:56.758117 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 12439
I20260806 21:09:56.758626 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:33975
--local_ip_for_outbound_sockets=127.27.225.194
--tserver_master_addrs=127.27.225.254:41497
--webserver_port=36011
--webserver_interface=127.27.225.194
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:40747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:09:56.766644 12485 log.cc:824] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2: Log is configured to *not* fsync() on all Append() calls
Aug 06 21:09:56 dist-test-slave-69rg krb5kdc[11803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050596, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:09:56.776026 12485 tablet_bootstrap.cc:492] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 21:09:56.776543 12485 tablet_bootstrap.cc:492] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2: Bootstrap complete.
I20260806 21:09:56.777580 12598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:56.757935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.193:45625)
0806 21:09:56.759452 (+ 1517us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:56.759513 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:56.760610 (+ 1097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:56.762278 (+ 1668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:56.762292 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:56.762869 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:56.763598 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:56.763631 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:56.765389 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:56.765392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:56.766627 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:56.766640 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:56.767628 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:56.767634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:56.767891 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:56.768659 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:56.768687 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:56.770908 (+ 2221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:56.773715 (+ 2807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:56.773724 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:56.773741 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:56.774284 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:56.774822 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:56.774825 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:56.774827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:56.775004 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:56.775779 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:56.775785 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:56.776067 (+ 282us) client_negotiation.cc:769] Sending connection context
0806 21:09:56.776249 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 21:09:56.776591 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1304,"thread_start_us":177,"threads_started":1}
I20260806 21:09:56.777743 12485 ts_tablet_manager.cc:1397] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2: Time spent bootstrapping tablet: real 0.034s user 0.028s sys 0.000s
I20260806 21:09:56.779625 12597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:56.757924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:45625 (local address 127.27.225.254:41497)
0806 21:09:56.759134 (+ 1210us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:56.759154 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:56.759641 (+ 487us) server_negotiation.cc:407] Connection header received
0806 21:09:56.761143 (+ 1502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:56.761162 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:56.761616 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:56.762056 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:56.763866 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:56.765133 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:56.766777 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:56.767447 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:56.767637 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:56.771170 (+ 3533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:56.771178 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:56.771193 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:56.771229 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:56.773567 (+ 2338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:56.774475 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:56.774483 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:56.774488 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:56.774684 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:56.775141 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:56.775146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:56.775148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:56.775645 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:56.775794 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:56.778385 (+ 2591us) server_negotiation.cc:299] Negotiation successful
0806 21:09:56.778790 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1050,"thread_start_us":145,"threads_started":1}
I20260806 21:09:56.780668 12485 raft_consensus.cc:348] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } } peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } }
I20260806 21:09:56.781240 12485 raft_consensus.cc:729] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05c8aa7600b14380a13ebc51085f91f2, State: Initialized, Role: FOLLOWER
I20260806 21:09:56.781741 12485 consensus_queue.cc:260] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } } peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } }
I20260806 21:09:56.782155 12595 heartbeater.cc:347] Connected to a master server at 127.27.225.254:41497
I20260806 21:09:56.782377 12595 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:56.782765 12485 ts_tablet_manager.cc:1428] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
I20260806 21:09:56.782927 12595 heartbeater.cc:511] Master 127.27.225.254:41497 requested a full tablet report, sending...
I20260806 21:09:56.785465 12396 ts_manager.cc:194] Registered new tserver with Master: 05c8aa7600b14380a13ebc51085f91f2 (127.27.225.193:32843)
I20260806 21:09:56.789019 12396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.193@KRBTEST.COM'} at 127.27.225.193:45625
I20260806 21:09:56.791740 12595 heartbeater.cc:503] Master 127.27.225.254:41497 was elected leader, sending a full tablet report...
Aug 06 21:09:56 dist-test-slave-69rg krb5kdc[11803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050596, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for kudu/127.27.225.193@KRBTEST.COM
I20260806 21:09:56.799603 12603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:56.788482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53284 (local address 127.27.225.193:32843)
0806 21:09:56.788919 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:56.788931 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:56.789350 (+ 419us) server_negotiation.cc:407] Connection header received
0806 21:09:56.789583 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:56.789589 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:56.789801 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:56.790003 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:56.790772 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:56.791800 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:56.792900 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:56.793246 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:56.793367 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:56.795759 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:56.795777 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:56.795787 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:56.795810 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:56.797422 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:56.797983 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:56.797989 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:56.797993 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:56.798116 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:56.798459 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:56.798464 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:56.798466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:56.798795 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:56.798937 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:56.799276 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 21:09:56.799442 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":121,"threads_started":1}
I20260806 21:09:56.799603 12602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:56.788368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:32843 (local address 127.0.0.1:53284)
0806 21:09:56.789183 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:56.789232 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:56.789387 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:56.790047 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:56.790053 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:56.790496 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:56.790630 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:56.790638 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:56.791950 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:56.791953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:56.792768 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:56.792778 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:56.793382 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:56.793386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:56.793508 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:56.794182 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:56.794193 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:56.795604 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:56.797547 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:56.797557 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:56.797562 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:56.797881 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:56.798267 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:56.798270 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:56.798272 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:56.798365 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:56.798910 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:56.798916 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:56.799050 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 21:09:56.799194 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 21:09:56.799414 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":128,"threads_started":1}
I20260806 21:09:56.800817 12529 tablet_service.cc:1514] Processing DeleteTablet for tablet fab4963c452142449f14603b62cc1222 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:09:36 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:53284
I20260806 21:09:56.801805 12604 tablet_replica.cc:330] stopping tablet replica
I20260806 21:09:56.802175 12604 raft_consensus.cc:2229] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:09:56.802446 12604 raft_consensus.cc:2258] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:09:56.803478 12604 ts_tablet_manager.cc:1905] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:09:56.807585 12604 ts_tablet_manager.cc:1918] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:09:56.807682 12604 log.cc:1196] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/wal/wals/fab4963c452142449f14603b62cc1222
I20260806 21:09:56.808010 12604 ts_tablet_manager.cc:1939] T fab4963c452142449f14603b62cc1222 P 05c8aa7600b14380a13ebc51085f91f2: Deleting consensus metadata
I20260806 21:09:56.810091 12386 catalog_manager.cc:4841] TS 05c8aa7600b14380a13ebc51085f91f2 (127.27.225.193:32843): tablet fab4963c452142449f14603b62cc1222 (table test-table [id=2a7d7f54fd5d493a82f9f7ff2ba22782]) successfully deleted
W20260806 21:09:56.883761 12599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:56.884063 12599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:56.884138 12599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:56.888237 12599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:56.888352 12599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:56.888445 12599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:09:56.893869 12599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:40747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:33975
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=36011
--tserver_master_addrs=127.27.225.254:41497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:56.894922 12599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:56.895891 12599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:56.905520 12610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:56.905496 12609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:56.905500 12612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:56.905982 12599 server_base.cc:1034] running on GCE node
I20260806 21:09:56.906437 12599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:56.907199 12599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:56.908397 12599 hybrid_clock.cc:648] HybridClock initialized: now 1786050596908372 us; error 47 us; skew 500 ppm
Aug 06 21:09:56 dist-test-slave-69rg krb5kdc[11803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050596, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:56.911228 12599 init.cc:377] Logged in from keytab as kudu/127.27.225.194@KRBTEST.COM (short username kudu)
I20260806 21:09:56.912161 12599 webserver.cc:460] Webserver started at http://127.27.225.194:36011/ using document root <none> and password file <none>
I20260806 21:09:56.912744 12599 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:56.912822 12599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:56.916653 12599 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260806 21:09:56.918841 12619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:56.920050 12599 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:09:56.920149 12599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "3000883cceea4af6a10cb3b59231f5d6"
format_stamp: "Formatted at 2026-08-06 21:09:35 on dist-test-slave-69rg"
server_key: "633da2a19380701e05ded2d340c14837"
server_key_iv: "d22e2c10032fec14e6853308a64d640b"
server_key_version: "encryptionkey@0"
I20260806 21:09:56.920671 12599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:56.947206 12599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:56.950042 12599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:56.950201 12599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:56.950743 12599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:09:56.951763 12627 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260806 21:09:56.955479 12627 ts_tablet_manager.cc:1852] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260806 21:09:56.955879 12627 ts_tablet_manager.cc:1905] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:09:56.959981 12627 ts_tablet_manager.cc:1918] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:09:56.960091 12627 log.cc:1196] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/wal/wals/fab4963c452142449f14603b62cc1222
I20260806 21:09:56.960444 12627 ts_tablet_manager.cc:1939] T fab4963c452142449f14603b62cc1222 P 3000883cceea4af6a10cb3b59231f5d6: Deleting consensus metadata
I20260806 21:09:56.960769 12599 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260806 21:09:56.960858 12599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.009s user 0.000s sys 0.000s
I20260806 21:09:56.960968 12599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:09:56.961017 12599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:56.970357 12599 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:33975
I20260806 21:09:56.970448 12735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:33975 every 8 connection(s)
I20260806 21:09:56.971261 12599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:09:56.980032 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 12599
I20260806 21:09:56.980479 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:40917
--local_ip_for_outbound_sockets=127.27.225.195
--tserver_master_addrs=127.27.225.254:41497
--webserver_port=38769
--webserver_interface=127.27.225.195
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:40747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 21:09:56 dist-test-slave-69rg krb5kdc[11803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050596, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:09:56.985675 12597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:56.973208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:60395 (local address 127.27.225.254:41497)
0806 21:09:56.973354 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:56.973358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:56.974313 (+ 955us) server_negotiation.cc:407] Connection header received
0806 21:09:56.975174 (+ 861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:56.975177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:56.975226 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:56.975306 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:56.976468 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:56.976996 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:56.978171 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:56.978399 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:56.978478 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:56.981339 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:56.981346 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:56.981352 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:56.981391 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:56.983231 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:56.983961 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:56.983964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:56.983965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:56.984046 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:56.984455 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:56.984457 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:56.984459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:56.984643 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:56.984721 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:56.985411 (+ 690us) server_negotiation.cc:299] Negotiation successful
0806 21:09:56.985545 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 21:09:56.987242 12738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:56.973605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.194:60395)
0806 21:09:56.974125 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:56.974167 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:56.974989 (+ 822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:56.975463 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:56.975470 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:56.975816 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:56.976329 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:56.976345 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:56.977105 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:56.977107 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:56.978057 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:56.978080 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:56.978598 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:56.978603 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:56.978763 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:56.979366 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:56.979383 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:56.981130 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:56.983367 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:56.983377 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:56.983400 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:56.983845 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:56.984143 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:56.984149 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:56.984154 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:56.984358 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:56.984753 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:56.984763 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:56.985120 (+ 357us) client_negotiation.cc:769] Sending connection context
0806 21:09:56.985369 (+ 249us) client_negotiation.cc:240] Negotiation successful
0806 21:09:56.985985 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":179,"threads_started":1}
I20260806 21:09:56.988682 12736 heartbeater.cc:347] Connected to a master server at 127.27.225.254:41497
I20260806 21:09:56.988948 12736 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:56.989444 12736 heartbeater.cc:511] Master 127.27.225.254:41497 requested a full tablet report, sending...
I20260806 21:09:56.990631 12396 ts_manager.cc:194] Registered new tserver with Master: 3000883cceea4af6a10cb3b59231f5d6 (127.27.225.194:33975)
I20260806 21:09:56.991540 12396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.194@KRBTEST.COM'} at 127.27.225.194:60395
W20260806 21:09:57.104537 12739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:09:57.104816 12739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:09:57.104861 12739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:09:57.109172 12739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:09:57.109293 12739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:09:57.109359 12739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:09:57.114920 12739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:40747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:40917
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=38769
--tserver_master_addrs=127.27.225.254:41497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:09:57.116041 12739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:09:57.117161 12739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:09:57.127239 12745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:57.127265 12747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:09:57.127247 12744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:09:57.127584 12739 server_base.cc:1034] running on GCE node
I20260806 21:09:57.128026 12739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:09:57.128684 12739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:09:57.129890 12739 hybrid_clock.cc:648] HybridClock initialized: now 1786050597129863 us; error 51 us; skew 500 ppm
Aug 06 21:09:57 dist-test-slave-69rg krb5kdc[11803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050597, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:09:57.132602 12739 init.cc:377] Logged in from keytab as kudu/127.27.225.195@KRBTEST.COM (short username kudu)
I20260806 21:09:57.133514 12739 webserver.cc:460] Webserver started at http://127.27.225.195:38769/ using document root <none> and password file <none>
I20260806 21:09:57.134142 12739 fs_manager.cc:359] Metadata directory not provided
I20260806 21:09:57.134254 12739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:09:57.138051 12739 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.003s
I20260806 21:09:57.140056 12754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:57.141317 12739 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:09:57.141462 12739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "5e1256e1d36549b7a872dc6b51e79166"
format_stamp: "Formatted at 2026-08-06 21:09:35 on dist-test-slave-69rg"
server_key: "859e13c0f4faf322b761dd9797334d2f"
server_key_iv: "5f2658a7d0ba22b3adb28778b8098451"
server_key_version: "encryptionkey@0"
I20260806 21:09:57.142076 12739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:09:57.159209 12739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:09:57.162318 12739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:09:57.162534 12739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:09:57.163141 12739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:09:57.163998 12739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:09:57.164068 12739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:57.164139 12739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:09:57.164180 12739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:09:57.173734 12739 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:40917
I20260806 21:09:57.173760 12869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:40917 every 8 connection(s)
I20260806 21:09:57.174844 12739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:09:57.179494 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 12739
Aug 06 21:09:57 dist-test-slave-69rg krb5kdc[11803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050597, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:09:57.188376 12874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:57.180209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50500 (local address 127.27.225.254:41497)
0806 21:09:57.180459 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:57.180464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:57.180632 (+ 168us) server_negotiation.cc:407] Connection header received
0806 21:09:57.180781 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:57.180784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:57.180832 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:57.180929 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:57.181684 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.182262 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.183296 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.183538 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.183654 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:57.185042 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:57.185046 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:57.185048 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:57.185093 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:57.186652 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:57.187097 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:57.187101 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:57.187103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:57.187190 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:57.187482 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:57.187486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:57.187488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:57.187683 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:57.187902 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:57.188044 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 21:09:57.188165 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":78,"threads_started":1}
I20260806 21:09:57.188601 12597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:57.176661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:34735 (local address 127.27.225.254:41497)
0806 21:09:57.176810 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:57.176814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:57.177896 (+ 1082us) server_negotiation.cc:407] Connection header received
0806 21:09:57.178777 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:57.178779 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:57.178835 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:57.178909 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:57.180326 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.180856 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.182020 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.182243 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.182360 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:57.184768 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:57.184772 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:57.184780 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:57.184800 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:57.186538 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:57.187068 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:57.187072 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:57.187074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:57.187153 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:57.187485 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:57.187489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:57.187491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:57.187690 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:57.187792 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:57.188298 (+ 506us) server_negotiation.cc:299] Negotiation successful
0806 21:09:57.188414 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60,"spinlock_wait_cycles":4992}
I20260806 21:09:57.189936 12872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:57.177001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.195:34735)
0806 21:09:57.177684 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:57.177726 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:57.178591 (+ 865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:57.179035 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:57.179041 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:57.179543 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:09:57.180182 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:57.180197 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.180986 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.180990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:57.181899 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:57.181908 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.182397 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.182401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:57.182564 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:57.183164 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:09:57.183179 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:09:57.184604 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:09:57.186657 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:57.186663 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:57.186677 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:57.186956 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:57.187244 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:09:57.187248 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:09:57.187250 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:09:57.187391 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:09:57.187813 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:09:57.187819 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:09:57.188063 (+ 244us) client_negotiation.cc:769] Sending connection context
0806 21:09:57.188248 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 21:09:57.188858 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":134,"threads_started":1}
I20260806 21:09:57.191103 12870 heartbeater.cc:347] Connected to a master server at 127.27.225.254:41497
I20260806 21:09:57.191349 12870 heartbeater.cc:464] Registering TS with master...
I20260806 21:09:57.191784 12870 heartbeater.cc:511] Master 127.27.225.254:41497 requested a full tablet report, sending...
I20260806 21:09:57.192691 12396 ts_manager.cc:194] Registered new tserver with Master: 5e1256e1d36549b7a872dc6b51e79166 (127.27.225.195:40917)
I20260806 21:09:57.193213 12396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.195@KRBTEST.COM'} at 127.27.225.195:34735
I20260806 21:09:57.202529 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:09:57.206940 12874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:57.203236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50504 (local address 127.27.225.254:41497)
0806 21:09:57.203394 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:57.203398 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:57.203642 (+ 244us) server_negotiation.cc:407] Connection header received
0806 21:09:57.203823 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:57.203826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:57.203967 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:57.204065 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:09:57.204286 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.204835 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.205665 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.206032 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.206116 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:57.206406 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:09:57.206550 (+ 144us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 21:09:57.206830 (+ 280us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50504: token expired
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 21:09:57.207123 12874 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50504: token expired
W20260806 21:09:57.207417 12321 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.27.225.254:41497)
I20260806 21:09:57.214915 12874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:57.207799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50510 (local address 127.27.225.254:41497)
0806 21:09:57.207912 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:57.207915 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:57.207927 (+ 12us) server_negotiation.cc:407] Connection header received
0806 21:09:57.208006 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:57.208007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:57.208044 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:57.208106 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:09:57.208737 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.209218 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.210175 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.210404 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.210484 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:57.211898 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:09:57.211901 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:09:57.211903 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:09:57.211926 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:09:57.213426 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:57.213862 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:57.213866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:57.213870 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:57.213951 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:09:57.214191 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:09:57.214193 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:09:57.214194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:09:57.214384 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:09:57.214481 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:57.214687 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 21:09:57.214791 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 21:09:57.217489 12396 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50510:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:09:57.220901 12396 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:09:57.229293 12529 tablet_service.cc:1467] Processing CreateTablet for tablet c9466f5d2aac49d789367e57bfe843a0 (DEFAULT_TABLE table=test-table [id=42c3b5be81064c52a427396ca30c42e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:09:57.230036 12529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9466f5d2aac49d789367e57bfe843a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:09:57.233170 12485 tablet_bootstrap.cc:492] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2: Bootstrap starting.
I20260806 21:09:57.233896 12602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:57.228218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:40917 (local address 127.0.0.1:52148)
0806 21:09:57.228446 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:57.228463 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:57.228549 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:57.229614 (+ 1065us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:57.229617 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:57.229634 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:09:57.229764 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:57.229771 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.231319 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.231326 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:57.232221 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:57.232231 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.233429 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.233434 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:57.233537 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:57.233590 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 21:09:57.233641 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 21:09:57.233728 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":40}
I20260806 21:09:57.234086 12485 tablet_bootstrap.cc:654] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2: Neither blocks nor log segments found. Creating new log.
I20260806 21:09:57.234570 12879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:57.228421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52148 (local address 127.27.225.195:40917)
0806 21:09:57.229080 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:57.229093 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:57.229128 (+ 35us) server_negotiation.cc:407] Connection header received
0806 21:09:57.229208 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:57.229213 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:57.229433 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:57.229938 (+ 505us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:09:57.230117 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.231178 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.232352 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.233290 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.234102 (+ 812us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:57.234257 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:57.234324 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 21:09:57.234429 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":203,"threads_started":1}
I20260806 21:09:57.234608 12881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:57.228684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:33975 (local address 127.0.0.1:46158)
0806 21:09:57.228959 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:57.228975 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:57.229065 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:57.229780 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:57.229783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:57.229801 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:09:57.229930 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:57.229937 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.231405 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.231409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:57.232588 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:57.232602 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.234253 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.234256 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:57.234346 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:57.234367 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 21:09:57.234416 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 21:09:57.234478 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":25,"thread_start_us":112,"threads_started":1}
I20260806 21:09:57.235702 12485 tablet_bootstrap.cc:492] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2: No bootstrap required, opened a new log
I20260806 21:09:57.235802 12485 ts_tablet_manager.cc:1397] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:09:57.236254 12485 raft_consensus.cc:348] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } } peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } }
I20260806 21:09:57.236387 12485 raft_consensus.cc:374] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:09:57.236424 12485 raft_consensus.cc:729] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05c8aa7600b14380a13ebc51085f91f2, State: Initialized, Role: FOLLOWER
I20260806 21:09:57.236555 12485 consensus_queue.cc:260] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } } peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } }
I20260806 21:09:57.237247 12485 ts_tablet_manager.cc:1428] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:09:57.237977 12882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:57.229066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46158 (local address 127.27.225.194:33975)
0806 21:09:57.229352 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:57.229362 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:57.229379 (+ 17us) server_negotiation.cc:407] Connection header received
0806 21:09:57.229438 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:57.229442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:57.229628 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:57.229807 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:09:57.230316 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.231194 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.232739 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.234118 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.237465 (+ 3347us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:57.237613 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:57.237672 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 21:09:57.237769 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":101,"threads_started":1}
I20260806 21:09:57.237913 12803 tablet_service.cc:1467] Processing CreateTablet for tablet c9466f5d2aac49d789367e57bfe843a0 (DEFAULT_TABLE table=test-table [id=42c3b5be81064c52a427396ca30c42e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:09:57.239111 12803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9466f5d2aac49d789367e57bfe843a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:09:57.239540 12669 tablet_service.cc:1467] Processing CreateTablet for tablet c9466f5d2aac49d789367e57bfe843a0 (DEFAULT_TABLE table=test-table [id=42c3b5be81064c52a427396ca30c42e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:09:57.240525 12669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9466f5d2aac49d789367e57bfe843a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:09:57.245803 12627 tablet_bootstrap.cc:492] T c9466f5d2aac49d789367e57bfe843a0 P 3000883cceea4af6a10cb3b59231f5d6: Bootstrap starting.
I20260806 21:09:57.247736 12884 tablet_bootstrap.cc:492] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166: Bootstrap starting.
I20260806 21:09:57.248567 12627 tablet_bootstrap.cc:654] T c9466f5d2aac49d789367e57bfe843a0 P 3000883cceea4af6a10cb3b59231f5d6: Neither blocks nor log segments found. Creating new log.
I20260806 21:09:57.249356 12627 log.cc:824] T c9466f5d2aac49d789367e57bfe843a0 P 3000883cceea4af6a10cb3b59231f5d6: Log is configured to *not* fsync() on all Append() calls
I20260806 21:09:57.250211 12884 tablet_bootstrap.cc:654] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166: Neither blocks nor log segments found. Creating new log.
I20260806 21:09:57.250928 12884 log.cc:824] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166: Log is configured to *not* fsync() on all Append() calls
I20260806 21:09:57.251492 12627 tablet_bootstrap.cc:492] T c9466f5d2aac49d789367e57bfe843a0 P 3000883cceea4af6a10cb3b59231f5d6: No bootstrap required, opened a new log
I20260806 21:09:57.251688 12627 ts_tablet_manager.cc:1397] T c9466f5d2aac49d789367e57bfe843a0 P 3000883cceea4af6a10cb3b59231f5d6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:09:57.252797 12884 tablet_bootstrap.cc:492] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166: No bootstrap required, opened a new log
I20260806 21:09:57.252997 12884 ts_tablet_manager.cc:1397] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 21:09:57.254761 12627 raft_consensus.cc:348] T c9466f5d2aac49d789367e57bfe843a0 P 3000883cceea4af6a10cb3b59231f5d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } } peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } }
I20260806 21:09:57.254979 12627 raft_consensus.cc:374] T c9466f5d2aac49d789367e57bfe843a0 P 3000883cceea4af6a10cb3b59231f5d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:09:57.255038 12627 raft_consensus.cc:729] T c9466f5d2aac49d789367e57bfe843a0 P 3000883cceea4af6a10cb3b59231f5d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3000883cceea4af6a10cb3b59231f5d6, State: Initialized, Role: FOLLOWER
I20260806 21:09:57.255517 12627 consensus_queue.cc:260] T c9466f5d2aac49d789367e57bfe843a0 P 3000883cceea4af6a10cb3b59231f5d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } } peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } }
I20260806 21:09:57.256222 12736 heartbeater.cc:503] Master 127.27.225.254:41497 was elected leader, sending a full tablet report...
I20260806 21:09:57.256060 12884 raft_consensus.cc:348] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } } peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } }
I20260806 21:09:57.256320 12884 raft_consensus.cc:374] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:09:57.256374 12884 raft_consensus.cc:729] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e1256e1d36549b7a872dc6b51e79166, State: Initialized, Role: FOLLOWER
I20260806 21:09:57.256494 12627 ts_tablet_manager.cc:1428] T c9466f5d2aac49d789367e57bfe843a0 P 3000883cceea4af6a10cb3b59231f5d6: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260806 21:09:57.256934 12596 tablet.cc:2369] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:09:57.256852 12884 consensus_queue.cc:260] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } } peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } }
I20260806 21:09:57.258121 12870 heartbeater.cc:503] Master 127.27.225.254:41497 was elected leader, sending a full tablet report...
I20260806 21:09:57.258349 12884 ts_tablet_manager.cc:1428] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260806 21:09:57.425776 12871 tablet.cc:2369] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 21:09:57.472530 12737 tablet.cc:2369] T c9466f5d2aac49d789367e57bfe843a0 P 3000883cceea4af6a10cb3b59231f5d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:09:57.501580 12888 raft_consensus.cc:482] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:09:57.501855 12888 raft_consensus.cc:504] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } } peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } }
I20260806 21:09:57.503180 12888 leader_election.cc:288] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3000883cceea4af6a10cb3b59231f5d6 (127.27.225.194:33975), 05c8aa7600b14380a13ebc51085f91f2 (127.27.225.193:32843)
I20260806 21:09:57.508013 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:57.503643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:56725 (local address 127.27.225.193:32843)
0806 21:09:57.503941 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:57.503948 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:57.503975 (+ 27us) server_negotiation.cc:407] Connection header received
0806 21:09:57.504037 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:57.504040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:57.504124 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:57.504218 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:09:57.504572 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.505447 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.506947 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.507451 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.507589 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:57.507779 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:57.507817 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 21:09:57.507888 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":118,"threads_started":1}
I20260806 21:09:57.508011 12872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:57.503487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:32843 (local address 127.27.225.195:56725)
0806 21:09:57.503637 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:57.503656 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:57.503770 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:57.504261 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:57.504265 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:57.504303 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:09:57.504445 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:57.504451 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.505620 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.505624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:57.506784 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:57.506796 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.507588 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.507592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:57.507690 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:57.507713 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 21:09:57.507776 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 21:09:57.507851 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":146,"spinlock_wait_cycles":26752}
I20260806 21:09:57.508167 12882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:57.503604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:52345 (local address 127.27.225.194:33975)
0806 21:09:57.503737 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:57.503742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:57.504292 (+ 550us) server_negotiation.cc:407] Connection header received
0806 21:09:57.504411 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:57.504414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:57.504459 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:57.504554 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:09:57.504804 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.505510 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.506916 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.507470 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.507603 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:57.507655 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:57.507951 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 21:09:57.508028 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 21:09:57.508188 12890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:57.503487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:33975 (local address 127.27.225.195:52345)
0806 21:09:57.504157 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:57.504168 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 21:09:57.504308 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:09:57.504594 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:09:57.504596 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:09:57.504611 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:09:57.504696 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:57.504700 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.505628 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.505630 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:57.506784 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:09:57.506794 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.507720 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.507723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:09:57.507818 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:57.507835 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 21:09:57.507930 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 21:09:57.508000 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"mutex_wait_us":494,"spinlock_wait_cycles":7168,"thread_start_us":105,"threads_started":1}
I20260806 21:09:57.508967 12690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9466f5d2aac49d789367e57bfe843a0" candidate_uuid: "5e1256e1d36549b7a872dc6b51e79166" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3000883cceea4af6a10cb3b59231f5d6" is_pre_election: true
I20260806 21:09:57.508957 12547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9466f5d2aac49d789367e57bfe843a0" candidate_uuid: "5e1256e1d36549b7a872dc6b51e79166" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05c8aa7600b14380a13ebc51085f91f2" is_pre_election: true
I20260806 21:09:57.509284 12547 raft_consensus.cc:2454] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e1256e1d36549b7a872dc6b51e79166 in term 0.
I20260806 21:09:57.509284 12690 raft_consensus.cc:2454] T c9466f5d2aac49d789367e57bfe843a0 P 3000883cceea4af6a10cb3b59231f5d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e1256e1d36549b7a872dc6b51e79166 in term 0.
I20260806 21:09:57.509855 12757 leader_election.cc:302] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 05c8aa7600b14380a13ebc51085f91f2, 3000883cceea4af6a10cb3b59231f5d6, 5e1256e1d36549b7a872dc6b51e79166; no voters:
I20260806 21:09:57.510154 12888 raft_consensus.cc:2781] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:09:57.510250 12888 raft_consensus.cc:482] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:09:57.510303 12888 raft_consensus.cc:3037] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:09:57.511408 12888 raft_consensus.cc:504] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } } peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } }
I20260806 21:09:57.511758 12888 leader_election.cc:288] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166 [CANDIDATE]: Term 1 election: Requested vote from peers 3000883cceea4af6a10cb3b59231f5d6 (127.27.225.194:33975), 05c8aa7600b14380a13ebc51085f91f2 (127.27.225.193:32843)
I20260806 21:09:57.512164 12690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9466f5d2aac49d789367e57bfe843a0" candidate_uuid: "5e1256e1d36549b7a872dc6b51e79166" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3000883cceea4af6a10cb3b59231f5d6"
I20260806 21:09:57.512174 12547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9466f5d2aac49d789367e57bfe843a0" candidate_uuid: "5e1256e1d36549b7a872dc6b51e79166" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05c8aa7600b14380a13ebc51085f91f2"
I20260806 21:09:57.512281 12690 raft_consensus.cc:3037] T c9466f5d2aac49d789367e57bfe843a0 P 3000883cceea4af6a10cb3b59231f5d6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:09:57.512374 12547 raft_consensus.cc:3037] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:09:57.513383 12690 raft_consensus.cc:2454] T c9466f5d2aac49d789367e57bfe843a0 P 3000883cceea4af6a10cb3b59231f5d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e1256e1d36549b7a872dc6b51e79166 in term 1.
I20260806 21:09:57.513468 12547 raft_consensus.cc:2454] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e1256e1d36549b7a872dc6b51e79166 in term 1.
I20260806 21:09:57.513794 12756 leader_election.cc:302] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3000883cceea4af6a10cb3b59231f5d6, 5e1256e1d36549b7a872dc6b51e79166; no voters:
I20260806 21:09:57.513995 12888 raft_consensus.cc:2781] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:09:57.514283 12888 raft_consensus.cc:686] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166 [term 1 LEADER]: Becoming Leader. State: Replica: 5e1256e1d36549b7a872dc6b51e79166, State: Running, Role: LEADER
I20260806 21:09:57.514662 12888 consensus_queue.cc:237] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } } peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } }
I20260806 21:09:57.517762 12399 catalog_manager.cc:5495] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e1256e1d36549b7a872dc6b51e79166 (127.27.225.195). New cstate: current_term: 1 leader_uuid: "5e1256e1d36549b7a872dc6b51e79166" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } health_report { overall_health: UNKNOWN } } }
I20260806 21:09:57.535672 12879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:57.532024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52158 (local address 127.27.225.195:40917)
0806 21:09:57.532170 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 21:09:57.532174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:09:57.532189 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:09:57.532330 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:09:57.532333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:09:57.532397 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:09:57.532506 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:09:57.532742 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.533260 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.534171 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:09:57.534408 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:09:57.534516 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:09:57.534845 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:09:57.535267 (+ 422us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:09:57.535402 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 21:09:57.535469 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 21:09:57.535530 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 21:09:57.541641 12690 raft_consensus.cc:1261] T c9466f5d2aac49d789367e57bfe843a0 P 3000883cceea4af6a10cb3b59231f5d6 [term 1 FOLLOWER]: Refusing update from remote peer 5e1256e1d36549b7a872dc6b51e79166: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:09:57.541674 12547 raft_consensus.cc:1261] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [term 1 FOLLOWER]: Refusing update from remote peer 5e1256e1d36549b7a872dc6b51e79166: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:09:57.542479 12891 consensus_queue.cc:1035] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:09:57.542680 12888 consensus_queue.cc:1035] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:09:57.559942 12396 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:09:57.560163 12396 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:09:57.562867 12396 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d: Sending DeleteTablet for 3 replicas of tablet c9466f5d2aac49d789367e57bfe843a0
I20260806 21:09:57.563712 12529 tablet_service.cc:1514] Processing DeleteTablet for tablet c9466f5d2aac49d789367e57bfe843a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:09:57 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:53284
I20260806 21:09:57.563814 12669 tablet_service.cc:1514] Processing DeleteTablet for tablet c9466f5d2aac49d789367e57bfe843a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:09:57 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:46158
I20260806 21:09:57.563935 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 12439
I20260806 21:09:57.564862 12903 tablet_replica.cc:330] T c9466f5d2aac49d789367e57bfe843a0 P 3000883cceea4af6a10cb3b59231f5d6: stopping tablet replica
I20260806 21:09:57.565243 12903 raft_consensus.cc:2229] T c9466f5d2aac49d789367e57bfe843a0 P 3000883cceea4af6a10cb3b59231f5d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:09:57.565593 12903 raft_consensus.cc:2258] T c9466f5d2aac49d789367e57bfe843a0 P 3000883cceea4af6a10cb3b59231f5d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:09:57.565784 12803 tablet_service.cc:1514] Processing DeleteTablet for tablet c9466f5d2aac49d789367e57bfe843a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:09:57 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:52148
I20260806 21:09:57.567215 12903 ts_tablet_manager.cc:1905] T c9466f5d2aac49d789367e57bfe843a0 P 3000883cceea4af6a10cb3b59231f5d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:09:57.569967 12903 ts_tablet_manager.cc:1918] T c9466f5d2aac49d789367e57bfe843a0 P 3000883cceea4af6a10cb3b59231f5d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:09:57.570108 12903 log.cc:1196] T c9466f5d2aac49d789367e57bfe843a0 P 3000883cceea4af6a10cb3b59231f5d6: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/wal/wals/c9466f5d2aac49d789367e57bfe843a0
I20260806 21:09:57.570096 12904 tablet_replica.cc:330] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166: stopping tablet replica
I20260806 21:09:57.570506 12903 ts_tablet_manager.cc:1939] T c9466f5d2aac49d789367e57bfe843a0 P 3000883cceea4af6a10cb3b59231f5d6: Deleting consensus metadata
I20260806 21:09:57.570462 12904 raft_consensus.cc:2229] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:09:57.570996 12904 raft_consensus.cc:2258] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:09:57.572201 12385 catalog_manager.cc:4841] TS 3000883cceea4af6a10cb3b59231f5d6 (127.27.225.194:33975): tablet c9466f5d2aac49d789367e57bfe843a0 (table test-table [id=42c3b5be81064c52a427396ca30c42e4]) successfully deleted
I20260806 21:09:57.572649 12904 ts_tablet_manager.cc:1905] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 21:09:57.574635 12386 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.225.193:32843 (error 108)
I20260806 21:09:57.574832 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 12599
I20260806 21:09:57.577250 12904 ts_tablet_manager.cc:1918] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:09:57.577351 12904 log.cc:1196] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/c9466f5d2aac49d789367e57bfe843a0
I20260806 21:09:57.577746 12904 ts_tablet_manager.cc:1939] T c9466f5d2aac49d789367e57bfe843a0 P 5e1256e1d36549b7a872dc6b51e79166: Deleting consensus metadata
I20260806 21:09:57.579167 12386 catalog_manager.cc:4841] TS 5e1256e1d36549b7a872dc6b51e79166 (127.27.225.195:40917): tablet c9466f5d2aac49d789367e57bfe843a0 (table test-table [id=42c3b5be81064c52a427396ca30c42e4]) successfully deleted
I20260806 21:09:57.580132 12881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:09:57.579786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:32843 (local address 127.0.0.1:53292)
0806 21:09:57.579945 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 21:09:57.580027 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.193:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260806 21:09:57.580384 12386 catalog_manager.cc:4584] TS 05c8aa7600b14380a13ebc51085f91f2 (127.27.225.193:32843): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c9466f5d2aac49d789367e57bfe843a0: Network error: Client connection negotiation failed: client connection to 127.27.225.193:32843: connect: Connection refused (error 111)
I20260806 21:09:57.581348 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 12739
I20260806 21:09:57.586867 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 12367
I20260806 21:10:17.594686 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:41497
--webserver_interface=127.27.225.254
--webserver_port=38293
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:40747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:41497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:10:17.712328 12906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:17.712683 12906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:17.712776 12906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:17.716964 12906 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 21:10:17.717075 12906 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 21:10:17.717101 12906 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 21:10:17.717127 12906 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:10:17.717170 12906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:17.717212 12906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:17.717254 12906 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:10:17.717290 12906 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:10:17.723173 12906 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:40747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:41497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:41497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=38293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:17.724408 12906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:17.725482 12906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:10:17.734856 12906 server_base.cc:1034] running on GCE node
W20260806 21:10:17.734844 12911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:17.734843 12912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:17.734840 12914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:17.735864 12906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:17.737016 12906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:17.738183 12906 hybrid_clock.cc:648] HybridClock initialized: now 1786050617738169 us; error 30 us; skew 500 ppm
Aug 06 21:10:17 dist-test-slave-69rg krb5kdc[11803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050617, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:17.745713 12906 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:10:17.746749 12906 webserver.cc:460] Webserver started at http://127.27.225.254:38293/ using document root <none> and password file <none>
I20260806 21:10:17.747305 12906 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:17.747370 12906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:17.751384 12906 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 21:10:17.753724 12921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:17.755270 12906 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:10:17.755402 12906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "387028fb73dc4583ad3c8fd99446462d"
format_stamp: "Formatted at 2026-08-06 21:09:35 on dist-test-slave-69rg"
server_key: "6b06e1265b88f856ac7443f108d68a25"
server_key_iv: "4cf774346be6934574001de52da924a8"
server_key_version: "encryptionkey@0"
I20260806 21:10:17.755990 12906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:17.773304 12906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:17.776919 12906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:17.777179 12906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:17.784958 12906 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:41497
I20260806 21:10:17.785307 12975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:41497 every 8 connection(s)
I20260806 21:10:17.786262 12906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:10:17.791116 12976 sys_catalog.cc:263] Verifying existing consensus state
I20260806 21:10:17.792955 12976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d: Bootstrap starting.
I20260806 21:10:17.793578 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 12906
I20260806 21:10:17.794119 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:32843
--local_ip_for_outbound_sockets=127.27.225.193
--tserver_master_addrs=127.27.225.254:41497
--webserver_port=33775
--webserver_interface=127.27.225.193
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:40747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:10:17.813365 12976 log.cc:824] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:17.826855 12976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 21:10:17.827359 12976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d: Bootstrap complete.
I20260806 21:10:17.832566 12976 raft_consensus.cc:348] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387028fb73dc4583ad3c8fd99446462d" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41497 } }
I20260806 21:10:17.833045 12976 raft_consensus.cc:729] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 387028fb73dc4583ad3c8fd99446462d, State: Initialized, Role: FOLLOWER
I20260806 21:10:17.833623 12976 consensus_queue.cc:260] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387028fb73dc4583ad3c8fd99446462d" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41497 } }
I20260806 21:10:17.833778 12976 raft_consensus.cc:388] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:10:17.833909 12976 raft_consensus.cc:482] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:10:17.834038 12976 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [term 2 FOLLOWER]: Advancing to term 3
I20260806 21:10:17.836192 12976 raft_consensus.cc:504] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387028fb73dc4583ad3c8fd99446462d" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41497 } }
I20260806 21:10:17.836566 12976 leader_election.cc:302] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 387028fb73dc4583ad3c8fd99446462d; no voters:
I20260806 21:10:17.836944 12976 leader_election.cc:288] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 21:10:17.837049 12980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [term 3 FOLLOWER]: Leader election won for term 3
I20260806 21:10:17.837273 12980 raft_consensus.cc:686] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [term 3 LEADER]: Becoming Leader. State: Replica: 387028fb73dc4583ad3c8fd99446462d, State: Running, Role: LEADER
I20260806 21:10:17.837666 12980 consensus_queue.cc:237] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387028fb73dc4583ad3c8fd99446462d" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41497 } }
I20260806 21:10:17.838392 12976 sys_catalog.cc:564] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:10:17.839497 12982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 387028fb73dc4583ad3c8fd99446462d. Latest consensus state: current_term: 3 leader_uuid: "387028fb73dc4583ad3c8fd99446462d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387028fb73dc4583ad3c8fd99446462d" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41497 } } }
I20260806 21:10:17.839506 12981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "387028fb73dc4583ad3c8fd99446462d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "387028fb73dc4583ad3c8fd99446462d" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41497 } } }
I20260806 21:10:17.839649 12982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:17.839663 12981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:17.840312 12986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:10:17.846547 12986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:10:17.847311 12986 catalog_manager.cc:1259] Loaded cluster ID: 32aac675b8a74103af16b25e6c1eb8ed
I20260806 21:10:17.847393 12986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:10:17.850131 12986 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:10:17.851058 12986 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d: Loaded TSK: 0
I20260806 21:10:17.851167 12986 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 387028fb73dc4583ad3c8fd99446462d: Loaded TSK: 1
F20260806 21:10:17.851989 12986 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f134451c9ad google::LogMessage::Fail() at ??:0
@ 0x7f134451f88a google::LogMessage::SendToLog() at ??:0
@ 0x7f134451c4de google::LogMessage::Flush() at ??:0
@ 0x7f134451da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f13471e018d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f13471d8f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f134ee877bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f134ee6199c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f134ee62178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f134eea3df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f134eec282b std::function<>::operator()() at ??:0
@ 0x7f134ee61b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f134ee62f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f134ee5f579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f134eea3640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f134676e3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f134676ec83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f13467703f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f134675f512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f1347da76db start_thread at ??:0
@ 0x7f134301a61f clone at ??:0
W20260806 21:10:17.956115 12978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:17.956544 12978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:17.956626 12978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:17.963133 12978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:17.963248 12978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:17.963469 12978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:10:17.972306 12978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:40747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:32843
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=33775
--tserver_master_addrs=127.27.225.254:41497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:17.973874 12978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:17.975338 12978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:17.988011 13025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:17.988160 13028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:17.987984 13024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:17.988044 12978 server_base.cc:1034] running on GCE node
I20260806 21:10:17.988796 12978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:17.989688 12978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:17.990939 12978 hybrid_clock.cc:648] HybridClock initialized: now 1786050617990913 us; error 40 us; skew 500 ppm
Aug 06 21:10:17 dist-test-slave-69rg krb5kdc[11803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050617, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:17.994601 12978 init.cc:377] Logged in from keytab as kudu/127.27.225.193@KRBTEST.COM (short username kudu)
I20260806 21:10:17.995626 12978 webserver.cc:460] Webserver started at http://127.27.225.193:33775/ using document root <none> and password file <none>
I20260806 21:10:17.996225 12978 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:17.996289 12978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:18.000702 12978 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 21:10:18.003443 13035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:18.005155 12978 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 21:10:18.005441 12978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "05c8aa7600b14380a13ebc51085f91f2"
format_stamp: "Formatted at 2026-08-06 21:09:35 on dist-test-slave-69rg"
server_key: "eb205018129a7e4c425e97d29baa443f"
server_key_iv: "ce63ffda53a082266852ee1d07ba2ae7"
server_key_version: "encryptionkey@0"
I20260806 21:10:18.006206 12978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:18.021737 12978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:18.025921 12978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:18.026162 12978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:18.026847 12978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:10:18.028033 13042 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 21:10:18.033675 12978 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 21:10:18.033794 12978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260806 21:10:18.033900 12978 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 21:10:18.035794 12978 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 21:10:18.035882 12978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 21:10:18.036203 13042 tablet_bootstrap.cc:492] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2: Bootstrap starting.
I20260806 21:10:18.047437 12978 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:32843
I20260806 21:10:18.047564 13153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:32843 every 8 connection(s)
I20260806 21:10:18.048584 12978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:10:18.054138 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 12978
I20260806 21:10:18.054824 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:33975
--local_ip_for_outbound_sockets=127.27.225.194
--tserver_master_addrs=127.27.225.254:41497
--webserver_port=36011
--webserver_interface=127.27.225.194
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:40747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:10:18.058784 13042 log.cc:824] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:18.072664 13042 tablet_bootstrap.cc:492] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 21:10:18.073138 13042 tablet_bootstrap.cc:492] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2: Bootstrap complete.
I20260806 21:10:18.074436 13042 ts_tablet_manager.cc:1397] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2: Time spent bootstrapping tablet: real 0.038s user 0.026s sys 0.008s
I20260806 21:10:18.077169 13042 raft_consensus.cc:348] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } } peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } }
I20260806 21:10:18.077689 13042 raft_consensus.cc:729] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05c8aa7600b14380a13ebc51085f91f2, State: Initialized, Role: FOLLOWER
I20260806 21:10:18.078308 13042 consensus_queue.cc:260] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } } peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } }
I20260806 21:10:18.079394 13042 ts_tablet_manager.cc:1428] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260806 21:10:18.142163 13156 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:10:18.051145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.193:37799)
0806 21:10:18.051728 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:18.051787 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:18.052989 (+ 1202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:18.141060 (+ 88071us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":180,"threads_started":1}
W20260806 21:10:18.142920 13154 heartbeater.cc:644] Failed to heartbeat to 127.27.225.254:41497 (0 consecutive failures): Network error: Failed to ping master at 127.27.225.254:41497: Client connection negotiation failed: client connection to 127.27.225.254:41497: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 21:10:18.143890 13156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:18.143555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.193:43919)
0806 21:10:18.143741 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:18.143820 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 21:10:18.144596 13156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:18.144377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.193:47925)
0806 21:10:18.144492 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:18.144543 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 21:10:18.144793 13154 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 21:10:18.203094 13157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:18.203478 13157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:18.203568 13157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:18.208631 13157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:18.208789 13157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:18.208899 13157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:10:18.215025 13157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:40747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:33975
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=36011
--tserver_master_addrs=127.27.225.254:41497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:18.216491 13157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:18.218043 13157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:18.229393 13165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:18.229522 13164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:18.229522 13167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:18.229878 13157 server_base.cc:1034] running on GCE node
I20260806 21:10:18.230355 13157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:18.231099 13157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:18.232295 13157 hybrid_clock.cc:648] HybridClock initialized: now 1786050618232275 us; error 34 us; skew 500 ppm
Aug 06 21:10:18 dist-test-slave-69rg krb5kdc[11803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050618, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:18.236086 13157 init.cc:377] Logged in from keytab as kudu/127.27.225.194@KRBTEST.COM (short username kudu)
I20260806 21:10:18.237284 13157 webserver.cc:460] Webserver started at http://127.27.225.194:36011/ using document root <none> and password file <none>
I20260806 21:10:18.238032 13157 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:18.238168 13157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:18.242893 13157 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260806 21:10:18.245893 13175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:18.247376 13157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 21:10:18.247560 13157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "3000883cceea4af6a10cb3b59231f5d6"
format_stamp: "Formatted at 2026-08-06 21:09:35 on dist-test-slave-69rg"
server_key: "633da2a19380701e05ded2d340c14837"
server_key_iv: "d22e2c10032fec14e6853308a64d640b"
server_key_version: "encryptionkey@0"
I20260806 21:10:18.248214 13157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:18.259521 13157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:18.263510 13157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:18.263788 13157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:18.264644 13157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:10:18.265956 13157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:10:18.266089 13157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:18.266152 13157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:10:18.266170 13157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:18.277367 13157 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:33975
I20260806 21:10:18.277383 13290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:33975 every 8 connection(s)
I20260806 21:10:18.278545 13157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:10:18.282454 13293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:18.280829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.194:44503)
0806 21:10:18.281579 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:18.281695 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":116,"threads_started":1}
W20260806 21:10:18.283017 13291 heartbeater.cc:644] Failed to heartbeat to 127.27.225.254:41497 (0 consecutive failures): Network error: Failed to ping master at 127.27.225.254:41497: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
I20260806 21:10:18.283833 13293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:18.283521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.194:39989)
0806 21:10:18.283689 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:18.283762 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 21:10:18.284547 13293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:18.284311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.194:55183)
0806 21:10:18.284431 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:18.284493 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 21:10:18.284765 13291 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 21:10:18.285591 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 13157
I20260806 21:10:18.286119 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:40917
--local_ip_for_outbound_sockets=127.27.225.195
--tserver_master_addrs=127.27.225.254:41497
--webserver_port=38769
--webserver_interface=127.27.225.195
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:40747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:10:18.405596 13294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:18.406041 13294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:18.406123 13294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:18.411273 13294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:18.411353 13294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:18.411446 13294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:10:18.417178 13294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:40747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:40917
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=38769
--tserver_master_addrs=127.27.225.254:41497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:18.418519 13294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:18.419538 13294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:18.429522 13300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:18.429512 13302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:18.429512 13299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:18.429629 13294 server_base.cc:1034] running on GCE node
I20260806 21:10:18.430207 13294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:18.430805 13294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:18.431967 13294 hybrid_clock.cc:648] HybridClock initialized: now 1786050618431939 us; error 40 us; skew 500 ppm
Aug 06 21:10:18 dist-test-slave-69rg krb5kdc[11803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050618, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:18.434916 13294 init.cc:377] Logged in from keytab as kudu/127.27.225.195@KRBTEST.COM (short username kudu)
I20260806 21:10:18.435783 13294 webserver.cc:460] Webserver started at http://127.27.225.195:38769/ using document root <none> and password file <none>
I20260806 21:10:18.436300 13294 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:18.436358 13294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:18.440279 13294 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:10:18.442509 13309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:18.443867 13294 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:10:18.444007 13294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "5e1256e1d36549b7a872dc6b51e79166"
format_stamp: "Formatted at 2026-08-06 21:09:35 on dist-test-slave-69rg"
server_key: "859e13c0f4faf322b761dd9797334d2f"
server_key_iv: "5f2658a7d0ba22b3adb28778b8098451"
server_key_version: "encryptionkey@0"
I20260806 21:10:18.444578 13294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:18.457477 13294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:18.461211 13294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:18.461444 13294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:18.462198 13294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:10:18.463086 13294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:10:18.463158 13294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:18.463223 13294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:10:18.463267 13294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:18.473014 13294 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:40917
I20260806 21:10:18.473083 13428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:40917 every 8 connection(s)
I20260806 21:10:18.474197 13294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:10:18.478204 13431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:18.476563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.195:47535)
0806 21:10:18.477272 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:18.477383 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":115,"threads_started":1}
W20260806 21:10:18.478695 13429 heartbeater.cc:644] Failed to heartbeat to 127.27.225.254:41497 (0 consecutive failures): Network error: Failed to ping master at 127.27.225.254:41497: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
I20260806 21:10:18.478820 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 13294
I20260806 21:10:18.479369 13431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:18.479132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.195:53403)
0806 21:10:18.479267 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:18.479315 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 21:10:18.480157 13431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:18.479950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.195:42203)
0806 21:10:18.480068 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:18.480108 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 21:10:18.480520 13429 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 21:10:19.146093 13434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:19.145508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.193:34807)
0806 21:10:19.145905 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:19.145974 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":86,"threads_started":1}
I20260806 21:10:19.286049 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:19.285476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.194:55309)
0806 21:10:19.285789 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:19.285911 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":120,"threads_started":1}
I20260806 21:10:19.482232 13437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:19.481565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.195:32911)
0806 21:10:19.482018 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:19.482088 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":104,"threads_started":1}
I20260806 21:10:19.571513 13438 raft_consensus.cc:482] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:10:19.571802 13438 raft_consensus.cc:504] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } } peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } }
I20260806 21:10:19.572991 13438 leader_election.cc:288] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e1256e1d36549b7a872dc6b51e79166 (127.27.225.195:40917), 3000883cceea4af6a10cb3b59231f5d6 (127.27.225.194:33975)
Aug 06 21:10:19 dist-test-slave-69rg krb5kdc[11803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050617, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for kudu/127.27.225.194@KRBTEST.COM
Aug 06 21:10:19 dist-test-slave-69rg krb5kdc[11803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050617, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for kudu/127.27.225.195@KRBTEST.COM
I20260806 21:10:19.587643 13434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:19.573186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:40917 (local address 127.27.225.193:44719)
0806 21:10:19.573333 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:19.573350 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:19.573459 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:19.575301 (+ 1842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:19.575306 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:19.575851 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:10:19.576328 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:19.576343 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:19.577609 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:19.577614 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:19.578653 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:19.578664 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:19.579495 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:19.579499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:19.579729 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:19.580446 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:10:19.580459 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:10:19.582439 (+ 1980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:10:19.584737 (+ 2298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:10:19.584740 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:10:19.584741 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:10:19.584994 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:10:19.585357 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:10:19.585360 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:10:19.585362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:10:19.585553 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:10:19.586420 (+ 867us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:10:19.586427 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:10:19.586782 (+ 355us) client_negotiation.cc:769] Sending connection context
0806 21:10:19.587006 (+ 224us) client_negotiation.cc:240] Negotiation successful
0806 21:10:19.587450 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":394,"spinlock_wait_cycles":29696}
I20260806 21:10:19.587679 13441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:19.573568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:45755 (local address 127.27.225.194:33975)
0806 21:10:19.573913 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:19.573932 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:19.573964 (+ 32us) server_negotiation.cc:407] Connection header received
0806 21:10:19.574597 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:19.574602 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:19.574957 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:19.575260 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:19.576505 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:19.577463 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:19.578811 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:19.579353 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:19.579546 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:19.582345 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:19.582356 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:19.582374 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:10:19.582412 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:19.584372 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:19.585221 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:19.585225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:19.585229 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:19.585382 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:19.585701 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:19.585705 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:19.585706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:19.586262 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:19.586420 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:19.587179 (+ 759us) server_negotiation.cc:299] Negotiation successful
0806 21:10:19.587489 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":58,"threads_started":1}
I20260806 21:10:19.587688 13440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:19.573395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:44719 (local address 127.27.225.195:40917)
0806 21:10:19.573740 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:19.573767 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:19.573838 (+ 71us) server_negotiation.cc:407] Connection header received
0806 21:10:19.574575 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:19.574582 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:19.574929 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:19.575254 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:19.576509 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:19.577462 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:19.578780 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:19.579371 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:19.579554 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:19.582628 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:19.582633 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:19.582642 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:10:19.582669 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:19.584621 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:19.585109 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:19.585115 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:19.585122 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:19.585256 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:19.585651 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:19.585654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:19.585656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:19.586193 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:19.586421 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:19.587179 (+ 758us) server_negotiation.cc:299] Negotiation successful
0806 21:10:19.587489 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":72,"threads_started":1}
I20260806 21:10:19.587644 13439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:19.573314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:33975 (local address 127.27.225.193:45755)
0806 21:10:19.573664 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:19.573677 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:19.573772 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:19.575483 (+ 1711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:19.575486 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:19.575854 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:10:19.576328 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:19.576343 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:19.577619 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:19.577622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:19.578628 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:19.578638 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:19.579495 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:19.579499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:19.579736 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:19.580398 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:10:19.580415 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:10:19.582091 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:10:19.584598 (+ 2507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:10:19.584604 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:10:19.584617 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:10:19.585004 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:10:19.585487 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:10:19.585489 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:10:19.585490 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:10:19.585566 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:10:19.586466 (+ 900us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:10:19.586468 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:10:19.586782 (+ 314us) client_negotiation.cc:769] Sending connection context
0806 21:10:19.587006 (+ 224us) client_negotiation.cc:240] Negotiation successful
0806 21:10:19.587397 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":251,"spinlock_wait_cycles":25344,"thread_start_us":72,"threads_started":1}
W20260806 21:10:19.590646 13038 leader_election.cc:341] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e1256e1d36549b7a872dc6b51e79166 (127.27.225.195:40917): Not found: Tablet not found: c9466f5d2aac49d789367e57bfe843a0
W20260806 21:10:19.591060 13037 leader_election.cc:341] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3000883cceea4af6a10cb3b59231f5d6 (127.27.225.194:33975): Not found: Tablet not found: c9466f5d2aac49d789367e57bfe843a0
I20260806 21:10:19.591197 13037 leader_election.cc:302] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05c8aa7600b14380a13ebc51085f91f2; no voters: 3000883cceea4af6a10cb3b59231f5d6, 5e1256e1d36549b7a872dc6b51e79166
I20260806 21:10:19.591441 13438 raft_consensus.cc:2726] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 21:10:20.147531 13442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:20.147004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.193:58245)
0806 21:10:20.147300 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:20.147415 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":93,"threads_started":1}
I20260806 21:10:20.287345 13443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:20.286897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.194:48987)
0806 21:10:20.287178 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:20.287243 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":95,"threads_started":1}
I20260806 21:10:20.483664 13444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:20.483239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.195:60365)
0806 21:10:20.483507 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:20.483567 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":79,"threads_started":1}
I20260806 21:10:21.148990 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:21.148531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.193:59207)
0806 21:10:21.148798 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:21.148887 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":83,"threads_started":1}
I20260806 21:10:21.288731 13446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:21.288228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.194:50799)
0806 21:10:21.288523 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:21.288616 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":86,"threads_started":1}
I20260806 21:10:21.485430 13447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:21.484778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.195:45033)
0806 21:10:21.485148 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:21.485235 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":115,"threads_started":1}
I20260806 21:10:21.547981 13448 raft_consensus.cc:482] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:10:21.548120 13448 raft_consensus.cc:504] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } } peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } }
I20260806 21:10:21.548666 13448 leader_election.cc:288] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e1256e1d36549b7a872dc6b51e79166 (127.27.225.195:40917), 3000883cceea4af6a10cb3b59231f5d6 (127.27.225.194:33975)
W20260806 21:10:21.549500 13038 leader_election.cc:341] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e1256e1d36549b7a872dc6b51e79166 (127.27.225.195:40917): Not found: Tablet not found: c9466f5d2aac49d789367e57bfe843a0
W20260806 21:10:21.549650 13037 leader_election.cc:341] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3000883cceea4af6a10cb3b59231f5d6 (127.27.225.194:33975): Not found: Tablet not found: c9466f5d2aac49d789367e57bfe843a0
I20260806 21:10:21.549741 13037 leader_election.cc:302] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05c8aa7600b14380a13ebc51085f91f2; no voters: 3000883cceea4af6a10cb3b59231f5d6, 5e1256e1d36549b7a872dc6b51e79166
I20260806 21:10:21.549983 13448 raft_consensus.cc:2726] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 21:10:22.150506 13449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:22.150029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.193:54627)
0806 21:10:22.150330 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:22.150394 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":106,"threads_started":1}
I20260806 21:10:22.290138 13451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:22.289600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.194:39381)
0806 21:10:22.289945 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:22.290012 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":85,"threads_started":1}
I20260806 21:10:22.486922 13452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:22.486375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.195:45277)
0806 21:10:22.486676 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:22.486738 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":83,"threads_started":1}
I20260806 21:10:23.152130 13453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:23.151586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.193:47119)
0806 21:10:23.151904 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:23.152006 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":78,"threads_started":1}
I20260806 21:10:23.291651 13454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:23.291164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.194:51799)
0806 21:10:23.291468 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:23.291540 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":83,"threads_started":1}
I20260806 21:10:23.488478 13455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:23.487927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.195:43433)
0806 21:10:23.488287 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:23.488358 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":87,"threads_started":1}
I20260806 21:10:24.153710 13456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:24.153071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.193:33271)
0806 21:10:24.153510 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:24.153590 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"spinlock_wait_cycles":166656,"thread_start_us":241,"threads_started":1}
I20260806 21:10:24.194602 13457 raft_consensus.cc:482] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:10:24.194743 13457 raft_consensus.cc:504] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } } peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } }
I20260806 21:10:24.195145 13457 leader_election.cc:288] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e1256e1d36549b7a872dc6b51e79166 (127.27.225.195:40917), 3000883cceea4af6a10cb3b59231f5d6 (127.27.225.194:33975)
W20260806 21:10:24.196182 13038 leader_election.cc:341] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e1256e1d36549b7a872dc6b51e79166 (127.27.225.195:40917): Not found: Tablet not found: c9466f5d2aac49d789367e57bfe843a0
W20260806 21:10:24.196408 13037 leader_election.cc:341] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3000883cceea4af6a10cb3b59231f5d6 (127.27.225.194:33975): Not found: Tablet not found: c9466f5d2aac49d789367e57bfe843a0
I20260806 21:10:24.196519 13037 leader_election.cc:302] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05c8aa7600b14380a13ebc51085f91f2; no voters: 3000883cceea4af6a10cb3b59231f5d6, 5e1256e1d36549b7a872dc6b51e79166
I20260806 21:10:24.196727 13457 raft_consensus.cc:2726] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 21:10:24.293434 13458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:24.292930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.194:43111)
0806 21:10:24.293266 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:24.293333 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":81,"threads_started":1}
I20260806 21:10:24.490224 13459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:24.489649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.195:42757)
0806 21:10:24.490044 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:24.490117 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":89,"threads_started":1}
I20260806 21:10:25.155347 13460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:25.154713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.193:53493)
0806 21:10:25.155140 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:25.155238 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":120,"threads_started":1}
I20260806 21:10:25.295315 13461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:25.294590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.194:33759)
0806 21:10:25.294986 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:25.295052 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":133,"threads_started":1}
I20260806 21:10:25.491899 13462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:25.491273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.195:49767)
0806 21:10:25.491692 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:25.491791 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":87,"threads_started":1}
I20260806 21:10:26.156849 13463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:26.156316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.193:57677)
0806 21:10:26.156679 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:26.156745 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":108,"threads_started":1}
I20260806 21:10:26.297168 13464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:26.296659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.194:52625)
0806 21:10:26.296988 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:26.297062 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":98,"threads_started":1}
I20260806 21:10:26.493209 13465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:26.492743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.195:39259)
0806 21:10:26.493046 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:26.493110 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":99,"threads_started":1}
I20260806 21:10:27.158308 13466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:27.157738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.193:57735)
0806 21:10:27.158125 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:27.158179 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":157,"threads_started":1}
I20260806 21:10:27.298971 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:27.298473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.194:59547)
0806 21:10:27.298801 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:27.298871 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":91,"threads_started":1}
I20260806 21:10:27.494529 13468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:27.494072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.195:48473)
0806 21:10:27.494367 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:27.494427 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":81,"threads_started":1}
I20260806 21:10:28.114159 13469 raft_consensus.cc:482] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:10:28.114334 13469 raft_consensus.cc:504] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1256e1d36549b7a872dc6b51e79166" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40917 } } peers { permanent_uuid: "3000883cceea4af6a10cb3b59231f5d6" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 33975 } } peers { permanent_uuid: "05c8aa7600b14380a13ebc51085f91f2" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 32843 } }
I20260806 21:10:28.114792 13469 leader_election.cc:288] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e1256e1d36549b7a872dc6b51e79166 (127.27.225.195:40917), 3000883cceea4af6a10cb3b59231f5d6 (127.27.225.194:33975)
W20260806 21:10:28.115759 13038 leader_election.cc:341] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e1256e1d36549b7a872dc6b51e79166 (127.27.225.195:40917): Not found: Tablet not found: c9466f5d2aac49d789367e57bfe843a0
W20260806 21:10:28.115882 13037 leader_election.cc:341] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3000883cceea4af6a10cb3b59231f5d6 (127.27.225.194:33975): Not found: Tablet not found: c9466f5d2aac49d789367e57bfe843a0
I20260806 21:10:28.115974 13037 leader_election.cc:302] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05c8aa7600b14380a13ebc51085f91f2; no voters: 3000883cceea4af6a10cb3b59231f5d6, 5e1256e1d36549b7a872dc6b51e79166
I20260806 21:10:28.116161 13469 raft_consensus.cc:2726] T c9466f5d2aac49d789367e57bfe843a0 P 05c8aa7600b14380a13ebc51085f91f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 21:10:28.159897 13470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:28.159361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.193:45633)
0806 21:10:28.159690 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:28.159753 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":122,"threads_started":1}
I20260806 21:10:28.300494 13471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:28.299937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.194:59759)
0806 21:10:28.300296 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:28.300365 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":90,"threads_started":1}
I20260806 21:10:28.495725 13472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:28.495334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.195:51413)
0806 21:10:28.495575 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:28.495629 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":77,"threads_started":1}
I20260806 21:10:29.161175 13473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:29.160726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.193:44037)
0806 21:10:29.161014 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:29.161076 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":81,"threads_started":1}
I20260806 21:10:29.302028 13474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:29.301510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.194:51309)
0806 21:10:29.301792 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:29.301927 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":87,"threads_started":1}
I20260806 21:10:29.497085 13475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:29.496612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.195:32929)
0806 21:10:29.496914 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:29.496984 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":86,"threads_started":1}
I20260806 21:10:30.162817 13476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:30.162259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.193:44591)
0806 21:10:30.162634 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:30.162709 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":90,"threads_started":1}
I20260806 21:10:30.303499 13477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:30.303044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.194:60499)
0806 21:10:30.303328 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:30.303399 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":93,"threads_started":1}
I20260806 21:10:30.498521 13478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:30.498070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.195:59231)
0806 21:10:30.498359 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:30.498422 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":79,"threads_started":1}
I20260806 21:10:31.164461 13479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:31.163825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.193:43745)
0806 21:10:31.164177 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:31.164255 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":92,"threads_started":1}
I20260806 21:10:31.305001 13480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:31.304515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.194:54109)
0806 21:10:31.304831 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:31.304900 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":95,"threads_started":1}
I20260806 21:10:31.500085 13481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:31.499552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.195:56785)
0806 21:10:31.499900 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:31.499977 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":118,"threads_started":1}
I20260806 21:10:32.165887 13482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:32.165418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.193:32769)
0806 21:10:32.165665 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:32.165722 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":78,"threads_started":1}
I20260806 21:10:32.306551 13483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:32.306074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.194:60357)
0806 21:10:32.306399 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:32.306451 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":76,"threads_started":1}
I20260806 21:10:32.501809 13484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:32.501255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.195:51307)
0806 21:10:32.501613 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:32.501692 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":102,"threads_started":1}
I20260806 21:10:33.167425 13485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:33.166974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.193:55557)
0806 21:10:33.167256 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:33.167325 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":75,"threads_started":1}
I20260806 21:10:33.307888 13486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:33.307470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41497 (local address 127.27.225.194:38777)
0806 21:10:33.307720 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:33.307792 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.254:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":80,"threads_started":1}
I20260806 21:10:33.481777 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 12978
I20260806 21:10:33.488929 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 13157
I20260806 21:10:33.495553 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 13294
2026-08-06T21:10:33Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59806 ms)
[----------] 30 tests from SecurityITest (255369 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:10:33 dist-test-slave-69rg krb5kdc[13489](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:10:33 dist-test-slave-69rg krb5kdc[13489](info): set up 2 sockets
Aug 06 21:10:33 dist-test-slave-69rg krb5kdc[13489](info): commencing operation
krb5kdc: starting...
W20260806 21:10:34.579664 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:10:34 dist-test-slave-69rg krb5kdc[13489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:10:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:10:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:10:34.726163 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:39121
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:44535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:39121
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:10:34.841042 13502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:34.841388 13502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:34.841470 13502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:34.845588 13502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:10:34.845711 13502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:34.845764 13502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:34.845803 13502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:10:34.845853 13502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:10:34.851352 13502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:44535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:39121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:39121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:34.852587 13502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:34.853701 13502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:10:34.862847 13502 server_base.cc:1034] running on GCE node
W20260806 21:10:34.862882 13510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:34.862936 13508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:34.863116 13507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:34.863839 13502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:34.865039 13502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:34.866245 13502 hybrid_clock.cc:648] HybridClock initialized: now 1786050634866212 us; error 48 us; skew 500 ppm
Aug 06 21:10:34 dist-test-slave-69rg krb5kdc[13489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050634, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:34.869297 13502 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:10:34.870276 13502 webserver.cc:460] Webserver started at http://127.27.225.254:43435/ using document root <none> and password file <none>
I20260806 21:10:34.870813 13502 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:34.870872 13502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:34.871037 13502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:34.873083 13502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "277423ee55144b98a0b4249d9e558bfb"
format_stamp: "Formatted at 2026-08-06 21:10:34 on dist-test-slave-69rg"
server_key: "0bf651f2a69dfd2fca483a9df24c1549"
server_key_iv: "a5d7e8084f2ee0bbde10eb47351056fb"
server_key_version: "encryptionkey@0"
I20260806 21:10:34.873696 13502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "277423ee55144b98a0b4249d9e558bfb"
format_stamp: "Formatted at 2026-08-06 21:10:34 on dist-test-slave-69rg"
server_key: "0bf651f2a69dfd2fca483a9df24c1549"
server_key_iv: "a5d7e8084f2ee0bbde10eb47351056fb"
server_key_version: "encryptionkey@0"
I20260806 21:10:34.877647 13502 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 21:10:34.880199 13517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:34.881459 13502 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:10:34.881572 13502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "277423ee55144b98a0b4249d9e558bfb"
format_stamp: "Formatted at 2026-08-06 21:10:34 on dist-test-slave-69rg"
server_key: "0bf651f2a69dfd2fca483a9df24c1549"
server_key_iv: "a5d7e8084f2ee0bbde10eb47351056fb"
server_key_version: "encryptionkey@0"
I20260806 21:10:34.881738 13502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:34.894821 13502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:34.898483 13502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:34.898701 13502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:34.906121 13502 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:39121
I20260806 21:10:34.906150 13570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:39121 every 8 connection(s)
I20260806 21:10:34.907100 13502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:10:34.910130 13571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:34.914124 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 13502
I20260806 21:10:34.914278 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:10:34.914575 28551 external_mini_cluster.cc:1428] Setting key 21dc7bd88cb7d705e06210b7d8663f63
I20260806 21:10:34.916044 13571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 277423ee55144b98a0b4249d9e558bfb: Bootstrap starting.
I20260806 21:10:34.918797 13571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 277423ee55144b98a0b4249d9e558bfb: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:34.919566 13571 log.cc:824] T 00000000000000000000000000000000 P 277423ee55144b98a0b4249d9e558bfb: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:34.921528 13571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 277423ee55144b98a0b4249d9e558bfb: No bootstrap required, opened a new log
Aug 06 21:10:34 dist-test-slave-69rg krb5kdc[13489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:10:34.926113 13571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 277423ee55144b98a0b4249d9e558bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "277423ee55144b98a0b4249d9e558bfb" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 39121 } }
I20260806 21:10:34.926353 13571 raft_consensus.cc:374] T 00000000000000000000000000000000 P 277423ee55144b98a0b4249d9e558bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:34.926414 13571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 277423ee55144b98a0b4249d9e558bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 277423ee55144b98a0b4249d9e558bfb, State: Initialized, Role: FOLLOWER
I20260806 21:10:34.926916 13571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 277423ee55144b98a0b4249d9e558bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "277423ee55144b98a0b4249d9e558bfb" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 39121 } }
I20260806 21:10:34.927074 13571 raft_consensus.cc:388] T 00000000000000000000000000000000 P 277423ee55144b98a0b4249d9e558bfb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:10:34.927165 13571 raft_consensus.cc:482] T 00000000000000000000000000000000 P 277423ee55144b98a0b4249d9e558bfb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:10:34.927268 13571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 277423ee55144b98a0b4249d9e558bfb [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:34.928427 13571 raft_consensus.cc:504] T 00000000000000000000000000000000 P 277423ee55144b98a0b4249d9e558bfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "277423ee55144b98a0b4249d9e558bfb" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 39121 } }
I20260806 21:10:34.928833 13571 leader_election.cc:302] T 00000000000000000000000000000000 P 277423ee55144b98a0b4249d9e558bfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 277423ee55144b98a0b4249d9e558bfb; no voters:
I20260806 21:10:34.929189 13571 leader_election.cc:288] T 00000000000000000000000000000000 P 277423ee55144b98a0b4249d9e558bfb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:10:34.929309 13576 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 277423ee55144b98a0b4249d9e558bfb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:10:34.929526 13576 raft_consensus.cc:686] T 00000000000000000000000000000000 P 277423ee55144b98a0b4249d9e558bfb [term 1 LEADER]: Becoming Leader. State: Replica: 277423ee55144b98a0b4249d9e558bfb, State: Running, Role: LEADER
I20260806 21:10:34.929958 13576 consensus_queue.cc:237] T 00000000000000000000000000000000 P 277423ee55144b98a0b4249d9e558bfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "277423ee55144b98a0b4249d9e558bfb" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 39121 } }
I20260806 21:10:34.930194 13574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:34.916074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33658 (local address 127.27.225.254:39121)
0806 21:10:34.916568 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:34.916584 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:34.916624 (+ 40us) server_negotiation.cc:407] Connection header received
0806 21:10:34.917483 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:34.917510 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:34.918019 (+ 509us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:34.918514 (+ 495us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:34.919352 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:34.920327 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:34.921188 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:34.921684 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:34.921887 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:34.924656 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:34.924662 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:34.924677 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:10:34.924706 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:34.926738 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:34.927256 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:34.927262 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:34.927268 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:34.927439 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:34.927773 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:34.927776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:34.927777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:34.928209 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:34.928365 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:34.928738 (+ 373us) server_negotiation.cc:299] Negotiation successful
0806 21:10:34.929080 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":154,"threads_started":1}
I20260806 21:10:34.930583 13571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 277423ee55144b98a0b4249d9e558bfb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:10:34.931167 13576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 277423ee55144b98a0b4249d9e558bfb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 277423ee55144b98a0b4249d9e558bfb. Latest consensus state: current_term: 1 leader_uuid: "277423ee55144b98a0b4249d9e558bfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "277423ee55144b98a0b4249d9e558bfb" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 39121 } } }
I20260806 21:10:34.931310 13576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 277423ee55144b98a0b4249d9e558bfb [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:34.932534 13582 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:10:34.933346 13577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 277423ee55144b98a0b4249d9e558bfb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "277423ee55144b98a0b4249d9e558bfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "277423ee55144b98a0b4249d9e558bfb" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 39121 } } }
I20260806 21:10:34.933454 13577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 277423ee55144b98a0b4249d9e558bfb [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:34.935452 13582 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:10:34.941085 13582 catalog_manager.cc:1347] Generated new cluster ID: e1e213c98be5483c9c5385ba9af6e604
I20260806 21:10:34.941161 13582 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:10:34.960752 13582 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:10:34.962235 13582 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:10:34.983125 13582 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 277423ee55144b98a0b4249d9e558bfb: Generated new TSK 0
I20260806 21:10:34.984169 13582 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.225.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
I20260806 21:10:35.039117 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:39121
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:44535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:10:35.158528 13598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:35.158879 13598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:35.158958 13598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:35.163116 13598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:35.163226 13598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:35.163327 13598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:10:35.168824 13598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:44535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:39121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:35.170464 13598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:35.171883 13598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:35.182179 13603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:35.182209 13604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:35.182260 13598 server_base.cc:1034] running on GCE node
W20260806 21:10:35.182192 13606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:35.182930 13598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:35.183589 13598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:35.184767 13598 hybrid_clock.cc:648] HybridClock initialized: now 1786050635184749 us; error 30 us; skew 500 ppm
Aug 06 21:10:35 dist-test-slave-69rg krb5kdc[13489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050635, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:35.187848 13598 init.cc:377] Logged in from keytab as kudu/127.27.225.193@KRBTEST.COM (short username kudu)
I20260806 21:10:35.188691 13598 webserver.cc:460] Webserver started at http://127.27.225.193:45893/ using document root <none> and password file <none>
I20260806 21:10:35.189226 13598 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:35.189285 13598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:35.189448 13598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:35.191526 13598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "ef04452be90340b194ec05ee82479d41"
format_stamp: "Formatted at 2026-08-06 21:10:35 on dist-test-slave-69rg"
server_key: "63569d50265078f789b0d54db5132cb0"
server_key_iv: "0f656592e4fe798d10a6b38ca62eee9d"
server_key_version: "encryptionkey@0"
I20260806 21:10:35.192067 13598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "ef04452be90340b194ec05ee82479d41"
format_stamp: "Formatted at 2026-08-06 21:10:35 on dist-test-slave-69rg"
server_key: "63569d50265078f789b0d54db5132cb0"
server_key_iv: "0f656592e4fe798d10a6b38ca62eee9d"
server_key_version: "encryptionkey@0"
I20260806 21:10:35.195990 13598 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 21:10:35.198748 13613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:35.200114 13598 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 21:10:35.200284 13598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "ef04452be90340b194ec05ee82479d41"
format_stamp: "Formatted at 2026-08-06 21:10:35 on dist-test-slave-69rg"
server_key: "63569d50265078f789b0d54db5132cb0"
server_key_iv: "0f656592e4fe798d10a6b38ca62eee9d"
server_key_version: "encryptionkey@0"
I20260806 21:10:35.200405 13598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:35.212067 13598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:35.215694 13598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:35.215921 13598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:35.216569 13598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:10:35.217461 13598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:10:35.217528 13598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:35.217595 13598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:10:35.217621 13598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:35.227341 13598 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:43697
I20260806 21:10:35.227447 13727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:43697 every 8 connection(s)
I20260806 21:10:35.228415 13598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:10:35.238116 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 13598
I20260806 21:10:35.238270 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:10:35.238602 28551 external_mini_cluster.cc:1428] Setting key 497cb77a0c7a52dda39aff679f39069a
Aug 06 21:10:35 dist-test-slave-69rg krb5kdc[13489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050635, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:10:35.243778 13574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:35.230453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:42737 (local address 127.27.225.254:39121)
0806 21:10:35.230590 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:35.230594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:35.231654 (+ 1060us) server_negotiation.cc:407] Connection header received
0806 21:10:35.233135 (+ 1481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:35.233139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:35.233200 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:35.233286 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:35.234542 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.235091 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.236260 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.236504 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.236614 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:35.239769 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:35.239778 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:35.239780 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:10:35.239805 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:35.241459 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:35.242092 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:35.242096 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:35.242097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:35.242188 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:35.242573 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:35.242576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:35.242578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:35.242772 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:35.242910 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:35.243445 (+ 535us) server_negotiation.cc:299] Negotiation successful
0806 21:10:35.243601 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 21:10:35.244699 13730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:35.230818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:39121 (local address 127.27.225.193:42737)
0806 21:10:35.231279 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:35.231481 (+ 202us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:35.232920 (+ 1439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:35.233462 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:35.233469 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:35.233897 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:10:35.234387 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:35.234401 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.235233 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.235236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:35.236122 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:35.236131 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.236674 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.236678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:35.236853 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:35.237482 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:10:35.237500 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:10:35.239601 (+ 2101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:10:35.241599 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:10:35.241606 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:10:35.241622 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:10:35.241982 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:10:35.242286 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:10:35.242290 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:10:35.242292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:10:35.242463 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:10:35.242906 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:10:35.242915 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:10:35.243184 (+ 269us) client_negotiation.cc:769] Sending connection context
0806 21:10:35.243387 (+ 203us) client_negotiation.cc:240] Negotiation successful
0806 21:10:35.243743 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":141,"threads_started":1}
I20260806 21:10:35.246132 13728 heartbeater.cc:347] Connected to a master server at 127.27.225.254:39121
I20260806 21:10:35.246409 13728 heartbeater.cc:464] Registering TS with master...
I20260806 21:10:35.246942 13728 heartbeater.cc:511] Master 127.27.225.254:39121 requested a full tablet report, sending...
I20260806 21:10:35.248746 13534 ts_manager.cc:194] Registered new tserver with Master: ef04452be90340b194ec05ee82479d41 (127.27.225.193:43697)
WARNING: no policy specified for kudu/127.27.225.194@KRBTEST.COM; defaulting to no policy
I20260806 21:10:35.250697 13534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.193@KRBTEST.COM'} at 127.27.225.193:42737
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
I20260806 21:10:35.274842 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:39121
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:44535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:10:35.392009 13733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:35.392279 13733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:35.392324 13733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:35.396435 13733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:35.396507 13733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:35.396567 13733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:10:35.402374 13733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:44535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:39121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:35.403493 13733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:35.404557 13733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:35.414601 13740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:35.414618 13742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:35.414592 13739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:35.414974 13733 server_base.cc:1034] running on GCE node
I20260806 21:10:35.415447 13733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:35.416191 13733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:35.417438 13733 hybrid_clock.cc:648] HybridClock initialized: now 1786050635417422 us; error 39 us; skew 500 ppm
Aug 06 21:10:35 dist-test-slave-69rg krb5kdc[13489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050635, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:35.420657 13733 init.cc:377] Logged in from keytab as kudu/127.27.225.194@KRBTEST.COM (short username kudu)
I20260806 21:10:35.421604 13733 webserver.cc:460] Webserver started at http://127.27.225.194:45923/ using document root <none> and password file <none>
I20260806 21:10:35.422338 13733 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:35.422434 13733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:35.422677 13733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:35.424861 13733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "5b4a9dcbca9c4189ba06d3cca010d0d5"
format_stamp: "Formatted at 2026-08-06 21:10:35 on dist-test-slave-69rg"
server_key: "7067a3ef83327c9d0dde3a834942703d"
server_key_iv: "240bbb1c6c6ddb91296070c11e63a5c0"
server_key_version: "encryptionkey@0"
I20260806 21:10:35.425458 13733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "5b4a9dcbca9c4189ba06d3cca010d0d5"
format_stamp: "Formatted at 2026-08-06 21:10:35 on dist-test-slave-69rg"
server_key: "7067a3ef83327c9d0dde3a834942703d"
server_key_iv: "240bbb1c6c6ddb91296070c11e63a5c0"
server_key_version: "encryptionkey@0"
I20260806 21:10:35.429384 13733 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260806 21:10:35.432190 13749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:35.433805 13733 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:10:35.434010 13733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "5b4a9dcbca9c4189ba06d3cca010d0d5"
format_stamp: "Formatted at 2026-08-06 21:10:35 on dist-test-slave-69rg"
server_key: "7067a3ef83327c9d0dde3a834942703d"
server_key_iv: "240bbb1c6c6ddb91296070c11e63a5c0"
server_key_version: "encryptionkey@0"
I20260806 21:10:35.434134 13733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:35.446696 13733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:35.450063 13733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:35.450277 13733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:35.450887 13733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:10:35.451779 13733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:10:35.451854 13733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:35.451927 13733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:10:35.451980 13733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:35.461485 13733 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:45553
I20260806 21:10:35.461504 13863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:45553 every 8 connection(s)
I20260806 21:10:35.462499 13733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:10:35.463104 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 13733
I20260806 21:10:35.463241 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:10:35.463521 28551 external_mini_cluster.cc:1428] Setting key 5a4d89c5a91856b727f410a963685a17
Aug 06 21:10:35 dist-test-slave-69rg krb5kdc[13489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050635, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.225.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.195@KRBTEST.COM" created.
I20260806 21:10:35.478968 13574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:35.464362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:49041 (local address 127.27.225.254:39121)
0806 21:10:35.464504 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:35.464510 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:35.465476 (+ 966us) server_negotiation.cc:407] Connection header received
0806 21:10:35.466926 (+ 1450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:35.466932 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:35.467000 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:35.467100 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:35.468810 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.469349 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.471361 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.471583 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.471692 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:35.474652 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:35.474656 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:35.474658 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:10:35.474684 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:35.476557 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:35.477101 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:35.477105 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:35.477107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:35.477196 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:35.477639 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:35.477642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:35.477644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:35.477902 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:35.478019 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:35.478655 (+ 636us) server_negotiation.cc:299] Negotiation successful
0806 21:10:35.478788 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 21:10:35.479918 13866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:35.464794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:39121 (local address 127.27.225.194:49041)
0806 21:10:35.465306 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:35.465359 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:35.466709 (+ 1350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:35.467321 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:35.467332 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:35.467909 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:10:35.468633 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:35.468667 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.469491 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.469498 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:35.471222 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:35.471238 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.471785 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.471793 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:35.472038 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:35.472693 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:10:35.472719 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:10:35.474459 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:10:35.476683 (+ 2224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:10:35.476689 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:10:35.476704 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:10:35.476970 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:10:35.477327 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:10:35.477330 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:10:35.477332 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:10:35.477521 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:10:35.478017 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:10:35.478024 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:10:35.478352 (+ 328us) client_negotiation.cc:769] Sending connection context
0806 21:10:35.478588 (+ 236us) client_negotiation.cc:240] Negotiation successful
0806 21:10:35.478937 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":161,"threads_started":1}
I20260806 21:10:35.481451 13864 heartbeater.cc:347] Connected to a master server at 127.27.225.254:39121
I20260806 21:10:35.481765 13864 heartbeater.cc:464] Registering TS with master...
I20260806 21:10:35.482465 13864 heartbeater.cc:511] Master 127.27.225.254:39121 requested a full tablet report, sending...
I20260806 21:10:35.483915 13534 ts_manager.cc:194] Registered new tserver with Master: 5b4a9dcbca9c4189ba06d3cca010d0d5 (127.27.225.194:45553)
I20260806 21:10:35.484524 13534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.194@KRBTEST.COM'} at 127.27.225.194:49041
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
I20260806 21:10:35.498010 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:39121
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:44535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:10:35.620249 13869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:35.620543 13869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:35.620620 13869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:35.624778 13869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:35.624873 13869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:35.624970 13869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:10:35.630606 13869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:44535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:39121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:35.631949 13869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:35.633226 13869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:35.643520 13875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:35.643666 13877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:35.643733 13869 server_base.cc:1034] running on GCE node
W20260806 21:10:35.643520 13874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:35.644164 13869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:35.644866 13869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:35.646062 13869 hybrid_clock.cc:648] HybridClock initialized: now 1786050635646022 us; error 48 us; skew 500 ppm
Aug 06 21:10:35 dist-test-slave-69rg krb5kdc[13489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050635, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:35.648931 13869 init.cc:377] Logged in from keytab as kudu/127.27.225.195@KRBTEST.COM (short username kudu)
I20260806 21:10:35.649947 13869 webserver.cc:460] Webserver started at http://127.27.225.195:45645/ using document root <none> and password file <none>
I20260806 21:10:35.650556 13869 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:35.650650 13869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:35.650897 13869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:35.653062 13869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "a26370f3e5e34ee798912a37c7256298"
format_stamp: "Formatted at 2026-08-06 21:10:35 on dist-test-slave-69rg"
server_key: "decb6a911ee8b1d673168f4e416409c0"
server_key_iv: "312742177b716f6217c18cf670d62f60"
server_key_version: "encryptionkey@0"
I20260806 21:10:35.653683 13869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "a26370f3e5e34ee798912a37c7256298"
format_stamp: "Formatted at 2026-08-06 21:10:35 on dist-test-slave-69rg"
server_key: "decb6a911ee8b1d673168f4e416409c0"
server_key_iv: "312742177b716f6217c18cf670d62f60"
server_key_version: "encryptionkey@0"
I20260806 21:10:35.657688 13869 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 21:10:35.660380 13884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:35.661746 13869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:10:35.661936 13869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "a26370f3e5e34ee798912a37c7256298"
format_stamp: "Formatted at 2026-08-06 21:10:35 on dist-test-slave-69rg"
server_key: "decb6a911ee8b1d673168f4e416409c0"
server_key_iv: "312742177b716f6217c18cf670d62f60"
server_key_version: "encryptionkey@0"
I20260806 21:10:35.662087 13869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:35.683030 13869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:35.686923 13869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:35.687144 13869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:35.687813 13869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:10:35.688683 13869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:10:35.688767 13869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:35.688838 13869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:10:35.688868 13869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:35.698417 13869 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:38665
I20260806 21:10:35.698451 14001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:38665 every 8 connection(s)
I20260806 21:10:35.699347 13869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:10:35.706161 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 13869
I20260806 21:10:35.706350 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:10:35.706696 28551 external_mini_cluster.cc:1428] Setting key f4e140bb34c29bfc593ca5646b4e23ea
Aug 06 21:10:35 dist-test-slave-69rg krb5kdc[13489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050635, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:10:35.714418 13574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:35.701204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:33947 (local address 127.27.225.254:39121)
0806 21:10:35.701369 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:35.701373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:35.702256 (+ 883us) server_negotiation.cc:407] Connection header received
0806 21:10:35.703797 (+ 1541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:35.703800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:35.703865 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:35.703986 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:35.705389 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.706180 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.707631 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.707866 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.707982 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:35.710632 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:35.710636 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:35.710638 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:10:35.710662 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:35.712297 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:35.712790 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:35.712792 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:35.712794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:35.712879 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:35.713205 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:35.713208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:35.713209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:35.713422 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:35.713514 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:35.714156 (+ 642us) server_negotiation.cc:299] Negotiation successful
0806 21:10:35.714284 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 21:10:35.715504 14004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:35.701557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:39121 (local address 127.27.225.195:33947)
0806 21:10:35.702085 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:35.702132 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:35.703499 (+ 1367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:35.704174 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:35.704182 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:35.704599 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:10:35.705229 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:35.705245 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.706347 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.706352 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:35.707494 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:35.707504 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.708024 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.708029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:35.708195 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:35.708876 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:10:35.708892 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:10:35.710459 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:10:35.712412 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:10:35.712417 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:10:35.712429 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:10:35.712685 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:10:35.712961 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:10:35.712963 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:10:35.712965 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:10:35.713094 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:10:35.713524 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:10:35.713529 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:10:35.713876 (+ 347us) client_negotiation.cc:769] Sending connection context
0806 21:10:35.714081 (+ 205us) client_negotiation.cc:240] Negotiation successful
0806 21:10:35.714356 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":124,"threads_started":1}
I20260806 21:10:35.716837 14002 heartbeater.cc:347] Connected to a master server at 127.27.225.254:39121
I20260806 21:10:35.717113 14002 heartbeater.cc:464] Registering TS with master...
I20260806 21:10:35.717654 14002 heartbeater.cc:511] Master 127.27.225.254:39121 requested a full tablet report, sending...
I20260806 21:10:35.718902 13534 ts_manager.cc:194] Registered new tserver with Master: a26370f3e5e34ee798912a37c7256298 (127.27.225.195:38665)
I20260806 21:10:35.719771 13534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.195@KRBTEST.COM'} at 127.27.225.195:33947
I20260806 21:10:35.721050 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:10:35.731745 13574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:35.723139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33660 (local address 127.27.225.254:39121)
0806 21:10:35.723291 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:35.723298 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:35.723479 (+ 181us) server_negotiation.cc:407] Connection header received
0806 21:10:35.723566 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:35.723568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:35.723629 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:35.723735 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:35.724522 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.725131 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.726201 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.726434 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.726551 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:35.728114 (+ 1563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:35.728118 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:35.728121 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:10:35.728161 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:35.729780 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:35.730264 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:35.730269 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:35.730271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:35.730361 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:35.730673 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:35.730677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:35.730679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:35.730989 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:35.731147 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:35.731405 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 21:10:35.731557 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 21:10:35.735276 13534 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33660:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:10:35.738758 13534 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:10:35.754376 14015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:35.748367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54224 (local address 127.27.225.193:43697)
0806 21:10:35.749638 (+ 1271us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:35.749646 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:35.749681 (+ 35us) server_negotiation.cc:407] Connection header received
0806 21:10:35.749763 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:35.749768 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:35.750006 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:35.750204 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:35.750612 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.751535 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.752985 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.753588 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.753750 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:35.753926 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:35.754101 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 21:10:35.754207 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1150,"thread_start_us":103,"threads_started":1}
I20260806 21:10:35.754362 14013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:35.748236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:43697 (local address 127.0.0.1:54224)
0806 21:10:35.748636 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:35.748696 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:35.748845 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:35.750364 (+ 1519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:35.750369 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:35.750392 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:35.750478 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:35.750483 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.751661 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.751666 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:35.752865 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:35.752877 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.753735 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.753739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:35.753881 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:35.753924 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 21:10:35.754066 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 21:10:35.754207 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":41,"thread_start_us":99,"threads_started":1}
I20260806 21:10:35.755508 14014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:35.748311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:45553 (local address 127.0.0.1:52076)
0806 21:10:35.748969 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:35.748984 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:35.749076 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:35.750207 (+ 1131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:35.750214 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:35.750241 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:35.750419 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:35.750433 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.752144 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.752147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:35.753563 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:35.753575 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.755054 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.755058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:35.755178 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:35.755201 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 21:10:35.755277 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 21:10:35.755354 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"mutex_wait_us":1,"thread_start_us":90,"threads_started":1}
I20260806 21:10:35.755620 14017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:35.749073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52076 (local address 127.27.225.194:45553)
0806 21:10:35.749739 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:35.749750 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:35.749768 (+ 18us) server_negotiation.cc:407] Connection header received
0806 21:10:35.749854 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:35.749858 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:35.750004 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:35.750204 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:35.750580 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.751952 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.753933 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.754928 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.755053 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:35.755162 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:35.755331 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 21:10:35.755453 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":172,"threads_started":1}
I20260806 21:10:35.755863 14018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:35.749479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36226 (local address 127.27.225.195:38665)
0806 21:10:35.750275 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:35.750298 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:35.750322 (+ 24us) server_negotiation.cc:407] Connection header received
0806 21:10:35.750401 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:35.750406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:35.750598 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:35.750773 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:35.751016 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.752215 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.753888 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.754847 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.754997 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:35.755136 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:35.755582 (+ 446us) server_negotiation.cc:299] Negotiation successful
0806 21:10:35.755694 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":173,"threads_started":1}
I20260806 21:10:35.756161 14016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:35.749010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:38665 (local address 127.0.0.1:36226)
0806 21:10:35.749260 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:35.749276 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:35.749375 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:35.750769 (+ 1394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:35.750771 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:35.750791 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:35.750867 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:35.750872 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.752370 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.752373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:35.753716 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:35.753725 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.755212 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.755216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:35.755324 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:35.755352 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 21:10:35.755430 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 21:10:35.755502 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":2,"thread_start_us":115,"threads_started":1}
I20260806 21:10:35.757964 13661 tablet_service.cc:1467] Processing CreateTablet for tablet e26636ea0ec64b45bb1dc2dcb42278cf (DEFAULT_TABLE table=test-table [id=9ca9c6b98ed84fab85d3ac1c103e8466]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:10:35.758803 13932 tablet_service.cc:1467] Processing CreateTablet for tablet e26636ea0ec64b45bb1dc2dcb42278cf (DEFAULT_TABLE table=test-table [id=9ca9c6b98ed84fab85d3ac1c103e8466]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:10:35.759195 13661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e26636ea0ec64b45bb1dc2dcb42278cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:35.758869 13797 tablet_service.cc:1467] Processing CreateTablet for tablet e26636ea0ec64b45bb1dc2dcb42278cf (DEFAULT_TABLE table=test-table [id=9ca9c6b98ed84fab85d3ac1c103e8466]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:10:35.759785 13932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e26636ea0ec64b45bb1dc2dcb42278cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:35.759833 13797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e26636ea0ec64b45bb1dc2dcb42278cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:35.766546 14019 tablet_bootstrap.cc:492] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298: Bootstrap starting.
I20260806 21:10:35.767395 14021 tablet_bootstrap.cc:492] T e26636ea0ec64b45bb1dc2dcb42278cf P ef04452be90340b194ec05ee82479d41: Bootstrap starting.
I20260806 21:10:35.769237 14021 tablet_bootstrap.cc:654] T e26636ea0ec64b45bb1dc2dcb42278cf P ef04452be90340b194ec05ee82479d41: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:35.769243 14019 tablet_bootstrap.cc:654] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:35.769711 14020 tablet_bootstrap.cc:492] T e26636ea0ec64b45bb1dc2dcb42278cf P 5b4a9dcbca9c4189ba06d3cca010d0d5: Bootstrap starting.
I20260806 21:10:35.770220 14021 log.cc:824] T e26636ea0ec64b45bb1dc2dcb42278cf P ef04452be90340b194ec05ee82479d41: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:35.770645 14019 log.cc:824] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:35.771981 14020 tablet_bootstrap.cc:654] T e26636ea0ec64b45bb1dc2dcb42278cf P 5b4a9dcbca9c4189ba06d3cca010d0d5: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:35.772481 14021 tablet_bootstrap.cc:492] T e26636ea0ec64b45bb1dc2dcb42278cf P ef04452be90340b194ec05ee82479d41: No bootstrap required, opened a new log
I20260806 21:10:35.772706 14021 ts_tablet_manager.cc:1397] T e26636ea0ec64b45bb1dc2dcb42278cf P ef04452be90340b194ec05ee82479d41: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 21:10:35.772840 14019 tablet_bootstrap.cc:492] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298: No bootstrap required, opened a new log
I20260806 21:10:35.773056 14019 ts_tablet_manager.cc:1397] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 21:10:35.773250 14020 log.cc:824] T e26636ea0ec64b45bb1dc2dcb42278cf P 5b4a9dcbca9c4189ba06d3cca010d0d5: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:35.775430 14020 tablet_bootstrap.cc:492] T e26636ea0ec64b45bb1dc2dcb42278cf P 5b4a9dcbca9c4189ba06d3cca010d0d5: No bootstrap required, opened a new log
I20260806 21:10:35.775616 14020 ts_tablet_manager.cc:1397] T e26636ea0ec64b45bb1dc2dcb42278cf P 5b4a9dcbca9c4189ba06d3cca010d0d5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:10:35.776549 14019 raft_consensus.cc:348] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef04452be90340b194ec05ee82479d41" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43697 } } peers { permanent_uuid: "5b4a9dcbca9c4189ba06d3cca010d0d5" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 45553 } } peers { permanent_uuid: "a26370f3e5e34ee798912a37c7256298" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38665 } }
I20260806 21:10:35.776822 14019 raft_consensus.cc:374] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:35.776880 14019 raft_consensus.cc:729] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a26370f3e5e34ee798912a37c7256298, State: Initialized, Role: FOLLOWER
I20260806 21:10:35.777251 14021 raft_consensus.cc:348] T e26636ea0ec64b45bb1dc2dcb42278cf P ef04452be90340b194ec05ee82479d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef04452be90340b194ec05ee82479d41" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43697 } } peers { permanent_uuid: "5b4a9dcbca9c4189ba06d3cca010d0d5" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 45553 } } peers { permanent_uuid: "a26370f3e5e34ee798912a37c7256298" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38665 } }
I20260806 21:10:35.777396 14019 consensus_queue.cc:260] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef04452be90340b194ec05ee82479d41" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43697 } } peers { permanent_uuid: "5b4a9dcbca9c4189ba06d3cca010d0d5" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 45553 } } peers { permanent_uuid: "a26370f3e5e34ee798912a37c7256298" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38665 } }
I20260806 21:10:35.777513 14021 raft_consensus.cc:374] T e26636ea0ec64b45bb1dc2dcb42278cf P ef04452be90340b194ec05ee82479d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:35.777570 14021 raft_consensus.cc:729] T e26636ea0ec64b45bb1dc2dcb42278cf P ef04452be90340b194ec05ee82479d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef04452be90340b194ec05ee82479d41, State: Initialized, Role: FOLLOWER
I20260806 21:10:35.778043 14021 consensus_queue.cc:260] T e26636ea0ec64b45bb1dc2dcb42278cf P ef04452be90340b194ec05ee82479d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef04452be90340b194ec05ee82479d41" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43697 } } peers { permanent_uuid: "5b4a9dcbca9c4189ba06d3cca010d0d5" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 45553 } } peers { permanent_uuid: "a26370f3e5e34ee798912a37c7256298" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38665 } }
I20260806 21:10:35.779001 14019 ts_tablet_manager.cc:1428] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 21:10:35.778997 14021 ts_tablet_manager.cc:1428] T e26636ea0ec64b45bb1dc2dcb42278cf P ef04452be90340b194ec05ee82479d41: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 21:10:35.779413 14002 heartbeater.cc:503] Master 127.27.225.254:39121 was elected leader, sending a full tablet report...
I20260806 21:10:35.779630 13728 heartbeater.cc:503] Master 127.27.225.254:39121 was elected leader, sending a full tablet report...
I20260806 21:10:35.779516 14020 raft_consensus.cc:348] T e26636ea0ec64b45bb1dc2dcb42278cf P 5b4a9dcbca9c4189ba06d3cca010d0d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef04452be90340b194ec05ee82479d41" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43697 } } peers { permanent_uuid: "5b4a9dcbca9c4189ba06d3cca010d0d5" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 45553 } } peers { permanent_uuid: "a26370f3e5e34ee798912a37c7256298" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38665 } }
I20260806 21:10:35.779770 14020 raft_consensus.cc:374] T e26636ea0ec64b45bb1dc2dcb42278cf P 5b4a9dcbca9c4189ba06d3cca010d0d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:35.779827 14020 raft_consensus.cc:729] T e26636ea0ec64b45bb1dc2dcb42278cf P 5b4a9dcbca9c4189ba06d3cca010d0d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b4a9dcbca9c4189ba06d3cca010d0d5, State: Initialized, Role: FOLLOWER
I20260806 21:10:35.780215 14020 consensus_queue.cc:260] T e26636ea0ec64b45bb1dc2dcb42278cf P 5b4a9dcbca9c4189ba06d3cca010d0d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef04452be90340b194ec05ee82479d41" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43697 } } peers { permanent_uuid: "5b4a9dcbca9c4189ba06d3cca010d0d5" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 45553 } } peers { permanent_uuid: "a26370f3e5e34ee798912a37c7256298" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38665 } }
I20260806 21:10:35.781129 13864 heartbeater.cc:503] Master 127.27.225.254:39121 was elected leader, sending a full tablet report...
I20260806 21:10:35.781386 14020 ts_tablet_manager.cc:1428] T e26636ea0ec64b45bb1dc2dcb42278cf P 5b4a9dcbca9c4189ba06d3cca010d0d5: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260806 21:10:35.862728 14025 raft_consensus.cc:482] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:10:35.862957 14025 raft_consensus.cc:504] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef04452be90340b194ec05ee82479d41" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43697 } } peers { permanent_uuid: "5b4a9dcbca9c4189ba06d3cca010d0d5" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 45553 } } peers { permanent_uuid: "a26370f3e5e34ee798912a37c7256298" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38665 } }
I20260806 21:10:35.864272 14025 leader_election.cc:288] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef04452be90340b194ec05ee82479d41 (127.27.225.193:43697), 5b4a9dcbca9c4189ba06d3cca010d0d5 (127.27.225.194:45553)
I20260806 21:10:35.869156 14017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:35.864801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:53393 (local address 127.27.225.194:45553)
0806 21:10:35.864927 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:35.864932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:35.865184 (+ 252us) server_negotiation.cc:407] Connection header received
0806 21:10:35.865339 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:35.865341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:35.865384 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:35.865477 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:35.865781 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.866338 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.867788 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.868385 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.868523 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:35.868577 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:35.868929 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 21:10:35.869019 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 21:10:35.869127 14004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:35.864499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:43697 (local address 127.27.225.195:40411)
0806 21:10:35.864657 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:35.864675 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:35.864781 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:35.865080 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:35.865083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:35.865109 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:35.865252 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:35.865259 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.866201 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.866207 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:35.867536 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:35.867547 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.868522 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.868526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:35.868705 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:35.868775 (+ 70us) client_negotiation.cc:769] Sending connection context
0806 21:10:35.868860 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 21:10:35.868957 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":63}
I20260806 21:10:35.869207 14015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:35.864658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:40411 (local address 127.27.225.193:43697)
0806 21:10:35.864793 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:35.864798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:35.864816 (+ 18us) server_negotiation.cc:407] Connection header received
0806 21:10:35.864889 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:35.864892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:35.864950 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:35.865034 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:35.865402 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.866090 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.867686 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.868387 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.868523 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:35.868577 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:35.868929 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 21:10:35.869034 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 21:10:35.869135 14028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:35.864644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:45553 (local address 127.27.225.195:53393)
0806 21:10:35.865083 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:35.865097 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:35.865186 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:35.865489 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:35.865492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:35.865510 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:35.865658 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:35.865664 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.866484 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.866487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:35.867656 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:35.867664 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.868522 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.868526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:35.868731 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:35.868778 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 21:10:35.868860 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 21:10:35.868957 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":88,"thread_start_us":136,"threads_started":1}
I20260806 21:10:35.870007 13681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e26636ea0ec64b45bb1dc2dcb42278cf" candidate_uuid: "a26370f3e5e34ee798912a37c7256298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef04452be90340b194ec05ee82479d41" is_pre_election: true
I20260806 21:10:35.870378 13681 raft_consensus.cc:2454] T e26636ea0ec64b45bb1dc2dcb42278cf P ef04452be90340b194ec05ee82479d41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a26370f3e5e34ee798912a37c7256298 in term 0.
I20260806 21:10:35.870994 13887 leader_election.cc:302] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a26370f3e5e34ee798912a37c7256298, ef04452be90340b194ec05ee82479d41; no voters:
I20260806 21:10:35.871197 13818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e26636ea0ec64b45bb1dc2dcb42278cf" candidate_uuid: "a26370f3e5e34ee798912a37c7256298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b4a9dcbca9c4189ba06d3cca010d0d5" is_pre_election: true
I20260806 21:10:35.871296 14025 raft_consensus.cc:2781] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:10:35.871425 14025 raft_consensus.cc:482] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:10:35.871536 14025 raft_consensus.cc:3037] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:35.871510 13818 raft_consensus.cc:2454] T e26636ea0ec64b45bb1dc2dcb42278cf P 5b4a9dcbca9c4189ba06d3cca010d0d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a26370f3e5e34ee798912a37c7256298 in term 0.
I20260806 21:10:35.872752 14025 raft_consensus.cc:504] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef04452be90340b194ec05ee82479d41" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43697 } } peers { permanent_uuid: "5b4a9dcbca9c4189ba06d3cca010d0d5" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 45553 } } peers { permanent_uuid: "a26370f3e5e34ee798912a37c7256298" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38665 } }
I20260806 21:10:35.873185 14025 leader_election.cc:288] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298 [CANDIDATE]: Term 1 election: Requested vote from peers ef04452be90340b194ec05ee82479d41 (127.27.225.193:43697), 5b4a9dcbca9c4189ba06d3cca010d0d5 (127.27.225.194:45553)
I20260806 21:10:35.873557 13818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e26636ea0ec64b45bb1dc2dcb42278cf" candidate_uuid: "a26370f3e5e34ee798912a37c7256298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b4a9dcbca9c4189ba06d3cca010d0d5"
I20260806 21:10:35.873528 13681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e26636ea0ec64b45bb1dc2dcb42278cf" candidate_uuid: "a26370f3e5e34ee798912a37c7256298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef04452be90340b194ec05ee82479d41"
I20260806 21:10:35.873690 13818 raft_consensus.cc:3037] T e26636ea0ec64b45bb1dc2dcb42278cf P 5b4a9dcbca9c4189ba06d3cca010d0d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:35.873690 13681 raft_consensus.cc:3037] T e26636ea0ec64b45bb1dc2dcb42278cf P ef04452be90340b194ec05ee82479d41 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:35.875174 13818 raft_consensus.cc:2454] T e26636ea0ec64b45bb1dc2dcb42278cf P 5b4a9dcbca9c4189ba06d3cca010d0d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a26370f3e5e34ee798912a37c7256298 in term 1.
I20260806 21:10:35.875173 13681 raft_consensus.cc:2454] T e26636ea0ec64b45bb1dc2dcb42278cf P ef04452be90340b194ec05ee82479d41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a26370f3e5e34ee798912a37c7256298 in term 1.
I20260806 21:10:35.875519 13887 leader_election.cc:302] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a26370f3e5e34ee798912a37c7256298, ef04452be90340b194ec05ee82479d41; no voters:
I20260806 21:10:35.875799 14025 raft_consensus.cc:2781] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:10:35.876060 14025 raft_consensus.cc:686] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298 [term 1 LEADER]: Becoming Leader. State: Replica: a26370f3e5e34ee798912a37c7256298, State: Running, Role: LEADER
I20260806 21:10:35.876439 14025 consensus_queue.cc:237] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef04452be90340b194ec05ee82479d41" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43697 } } peers { permanent_uuid: "5b4a9dcbca9c4189ba06d3cca010d0d5" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 45553 } } peers { permanent_uuid: "a26370f3e5e34ee798912a37c7256298" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38665 } }
I20260806 21:10:35.880081 13533 catalog_manager.cc:5495] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298 reported cstate change: term changed from 0 to 1, leader changed from <none> to a26370f3e5e34ee798912a37c7256298 (127.27.225.195). New cstate: current_term: 1 leader_uuid: "a26370f3e5e34ee798912a37c7256298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef04452be90340b194ec05ee82479d41" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b4a9dcbca9c4189ba06d3cca010d0d5" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 45553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a26370f3e5e34ee798912a37c7256298" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 38665 } health_report { overall_health: HEALTHY } } }
I20260806 21:10:35.909327 14018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:35.905364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36240 (local address 127.27.225.195:38665)
0806 21:10:35.905523 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:35.905528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:35.905543 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:10:35.905766 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:35.905770 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:35.905864 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:35.905976 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:10:35.906236 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.906930 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.907763 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:35.908066 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:35.908184 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:35.908440 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:10:35.908913 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:10:35.909037 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:35.909092 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 21:10:35.909161 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 21:10:35.916114 13681 raft_consensus.cc:1261] T e26636ea0ec64b45bb1dc2dcb42278cf P ef04452be90340b194ec05ee82479d41 [term 1 FOLLOWER]: Refusing update from remote peer a26370f3e5e34ee798912a37c7256298: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:10:35.916114 13818 raft_consensus.cc:1261] T e26636ea0ec64b45bb1dc2dcb42278cf P 5b4a9dcbca9c4189ba06d3cca010d0d5 [term 1 FOLLOWER]: Refusing update from remote peer a26370f3e5e34ee798912a37c7256298: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:10:35.916936 14025 consensus_queue.cc:1035] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b4a9dcbca9c4189ba06d3cca010d0d5" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:10:35.917176 14029 consensus_queue.cc:1035] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef04452be90340b194ec05ee82479d41" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 43697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:10:35.924571 14032 mvcc.cc:205] Tried to move back new op lower bound from 7315663404704141312 to 7315663404553138176. Current Snapshot: MvccSnapshot[applied={T|T < 7315663404704141312}]
I20260806 21:10:35.925573 14034 mvcc.cc:205] Tried to move back new op lower bound from 7315663404704141312 to 7315663404553138176. Current Snapshot: MvccSnapshot[applied={T|T < 7315663404704141312}]
I20260806 21:10:35.935518 13533 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:10:35.935719 13533 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:10:35.938383 13533 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 277423ee55144b98a0b4249d9e558bfb: Sending DeleteTablet for 3 replicas of tablet e26636ea0ec64b45bb1dc2dcb42278cf
I20260806 21:10:35.939294 13661 tablet_service.cc:1514] Processing DeleteTablet for tablet e26636ea0ec64b45bb1dc2dcb42278cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:10:35 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:54224
I20260806 21:10:35.939421 13932 tablet_service.cc:1514] Processing DeleteTablet for tablet e26636ea0ec64b45bb1dc2dcb42278cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:10:35 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:36226
I20260806 21:10:35.939894 13797 tablet_service.cc:1514] Processing DeleteTablet for tablet e26636ea0ec64b45bb1dc2dcb42278cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:10:35 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:52076
I20260806 21:10:35.940141 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 13598
I20260806 21:10:35.940430 14044 tablet_replica.cc:330] T e26636ea0ec64b45bb1dc2dcb42278cf P 5b4a9dcbca9c4189ba06d3cca010d0d5: stopping tablet replica
I20260806 21:10:35.940814 14044 raft_consensus.cc:2229] T e26636ea0ec64b45bb1dc2dcb42278cf P 5b4a9dcbca9c4189ba06d3cca010d0d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:10:35.940864 14042 tablet_replica.cc:330] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298: stopping tablet replica
I20260806 21:10:35.941130 14042 raft_consensus.cc:2229] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:10:35.941351 14044 raft_consensus.cc:2258] T e26636ea0ec64b45bb1dc2dcb42278cf P 5b4a9dcbca9c4189ba06d3cca010d0d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:10:35.941515 14042 raft_consensus.cc:2258] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:10:35.943136 14042 ts_tablet_manager.cc:1905] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:10:35.943136 14044 ts_tablet_manager.cc:1905] T e26636ea0ec64b45bb1dc2dcb42278cf P 5b4a9dcbca9c4189ba06d3cca010d0d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:10:35.946125 14044 ts_tablet_manager.cc:1918] T e26636ea0ec64b45bb1dc2dcb42278cf P 5b4a9dcbca9c4189ba06d3cca010d0d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:10:35.946125 14042 ts_tablet_manager.cc:1918] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:10:35.946242 14042 log.cc:1196] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/e26636ea0ec64b45bb1dc2dcb42278cf
I20260806 21:10:35.946391 14044 log.cc:1196] T e26636ea0ec64b45bb1dc2dcb42278cf P 5b4a9dcbca9c4189ba06d3cca010d0d5: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050378084738-28551-0/minicluster-data/ts-1/wal/wals/e26636ea0ec64b45bb1dc2dcb42278cf
I20260806 21:10:35.946623 14042 ts_tablet_manager.cc:1939] T e26636ea0ec64b45bb1dc2dcb42278cf P a26370f3e5e34ee798912a37c7256298: Deleting consensus metadata
I20260806 21:10:35.946867 14044 ts_tablet_manager.cc:1939] T e26636ea0ec64b45bb1dc2dcb42278cf P 5b4a9dcbca9c4189ba06d3cca010d0d5: Deleting consensus metadata
I20260806 21:10:35.948124 13521 catalog_manager.cc:4841] TS 5b4a9dcbca9c4189ba06d3cca010d0d5 (127.27.225.194:45553): tablet e26636ea0ec64b45bb1dc2dcb42278cf (table test-table [id=9ca9c6b98ed84fab85d3ac1c103e8466]) successfully deleted
I20260806 21:10:35.948544 13518 catalog_manager.cc:4841] TS a26370f3e5e34ee798912a37c7256298 (127.27.225.195:38665): tablet e26636ea0ec64b45bb1dc2dcb42278cf (table test-table [id=9ca9c6b98ed84fab85d3ac1c103e8466]) successfully deleted
W20260806 21:10:35.951911 13520 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.225.193:43697 (error 108)
I20260806 21:10:35.952811 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 13733
I20260806 21:10:35.953042 14016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:35.952736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:43697 (local address 127.0.0.1:54240)
0806 21:10:35.952892 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:35.952960 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.193:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260806 21:10:35.954803 13520 catalog_manager.cc:4584] TS ef04452be90340b194ec05ee82479d41 (127.27.225.193:43697): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e26636ea0ec64b45bb1dc2dcb42278cf: Network error: Client connection negotiation failed: client connection to 127.27.225.193:43697: connect: Connection refused (error 111)
I20260806 21:10:35.958902 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 13869
I20260806 21:10:35.964444 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 13502
2026-08-06T21:10:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2468 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T21:10:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:10:35Z Disabled control of system clock
I20260806 21:10:35.997742 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:42573
--webserver_interface=127.27.225.254
--webserver_port=0
--builtin_ntp_servers=127.27.225.212:35231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:42573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 21:10:36.118486 14049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:36.118774 14049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:36.118811 14049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:36.122931 14049 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:10:36.122994 14049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:36.123010 14049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:36.123028 14049 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:10:36.123097 14049 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:10:36.128187 14049 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:35231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:42573
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:42573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:36.129365 14049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:36.130828 14049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:36.139456 14054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:36.139456 14055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:36.139601 14049 server_base.cc:1034] running on GCE node
W20260806 21:10:36.139815 14057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:36.140410 14049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:36.141595 14049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:36.142805 14049 hybrid_clock.cc:648] HybridClock initialized: now 1786050636142795 us; error 32 us; skew 500 ppm
I20260806 21:10:36.144802 14049 webserver.cc:460] Webserver started at http://127.27.225.254:34103/ using document root <none> and password file <none>
I20260806 21:10:36.145454 14049 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:36.145553 14049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:36.145798 14049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:36.147933 14049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "ae77f322ee1b468c8f709e0e4a0bce01"
format_stamp: "Formatted at 2026-08-06 21:10:36 on dist-test-slave-69rg"
server_key: "49facd2dcbba700e1f5717116f1efa98"
server_key_iv: "526f65f790e6a9ef868bbcbc7df0a410"
server_key_version: "encryptionkey@0"
I20260806 21:10:36.148546 14049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "ae77f322ee1b468c8f709e0e4a0bce01"
format_stamp: "Formatted at 2026-08-06 21:10:36 on dist-test-slave-69rg"
server_key: "49facd2dcbba700e1f5717116f1efa98"
server_key_iv: "526f65f790e6a9ef868bbcbc7df0a410"
server_key_version: "encryptionkey@0"
I20260806 21:10:36.152376 14049 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:10:36.155176 14064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:36.156551 14049 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 21:10:36.156738 14049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "ae77f322ee1b468c8f709e0e4a0bce01"
format_stamp: "Formatted at 2026-08-06 21:10:36 on dist-test-slave-69rg"
server_key: "49facd2dcbba700e1f5717116f1efa98"
server_key_iv: "526f65f790e6a9ef868bbcbc7df0a410"
server_key_version: "encryptionkey@0"
I20260806 21:10:36.156878 14049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:36.178529 14049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:36.179343 14049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:36.179576 14049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:36.187907 14049 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:42573
I20260806 21:10:36.187913 14116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:42573 every 8 connection(s)
I20260806 21:10:36.188987 14049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:10:36.192451 14117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:36.194805 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 14049
I20260806 21:10:36.194963 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:10:36.195318 28551 external_mini_cluster.cc:1428] Setting key 63d0e707e1905a24357d3d3b4534d0b2
I20260806 21:10:36.198769 14117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae77f322ee1b468c8f709e0e4a0bce01: Bootstrap starting.
I20260806 21:10:36.201468 14117 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae77f322ee1b468c8f709e0e4a0bce01: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:36.202474 14117 log.cc:824] T 00000000000000000000000000000000 P ae77f322ee1b468c8f709e0e4a0bce01: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:36.204909 14117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae77f322ee1b468c8f709e0e4a0bce01: No bootstrap required, opened a new log
I20260806 21:10:36.205425 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:36.196960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47988 (local address 127.27.225.254:42573)
0806 21:10:36.197626 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:36.197642 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:36.197677 (+ 35us) server_negotiation.cc:407] Connection header received
0806 21:10:36.198373 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:36.198389 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:36.198781 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:36.199092 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:36.199458 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.200419 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.201685 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.202272 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.202418 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:36.203369 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:36.203376 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:36.203396 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:36.203846 (+ 450us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:36.203908 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:36.203930 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:36.204011 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:36.204230 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 21:10:36.204592 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":166,"threads_started":1}
I20260806 21:10:36.208809 14117 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae77f322ee1b468c8f709e0e4a0bce01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae77f322ee1b468c8f709e0e4a0bce01" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42573 } }
I20260806 21:10:36.209136 14117 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae77f322ee1b468c8f709e0e4a0bce01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:36.209241 14117 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae77f322ee1b468c8f709e0e4a0bce01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae77f322ee1b468c8f709e0e4a0bce01, State: Initialized, Role: FOLLOWER
I20260806 21:10:36.209719 14117 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae77f322ee1b468c8f709e0e4a0bce01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae77f322ee1b468c8f709e0e4a0bce01" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42573 } }
I20260806 21:10:36.209949 14117 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae77f322ee1b468c8f709e0e4a0bce01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:10:36.210021 14117 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae77f322ee1b468c8f709e0e4a0bce01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:10:36.210144 14117 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae77f322ee1b468c8f709e0e4a0bce01 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:36.211184 14117 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae77f322ee1b468c8f709e0e4a0bce01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae77f322ee1b468c8f709e0e4a0bce01" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42573 } }
I20260806 21:10:36.211560 14117 leader_election.cc:302] T 00000000000000000000000000000000 P ae77f322ee1b468c8f709e0e4a0bce01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae77f322ee1b468c8f709e0e4a0bce01; no voters:
I20260806 21:10:36.211923 14117 leader_election.cc:288] T 00000000000000000000000000000000 P ae77f322ee1b468c8f709e0e4a0bce01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:10:36.212077 14122 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae77f322ee1b468c8f709e0e4a0bce01 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:10:36.212313 14122 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae77f322ee1b468c8f709e0e4a0bce01 [term 1 LEADER]: Becoming Leader. State: Replica: ae77f322ee1b468c8f709e0e4a0bce01, State: Running, Role: LEADER
I20260806 21:10:36.212662 14122 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae77f322ee1b468c8f709e0e4a0bce01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae77f322ee1b468c8f709e0e4a0bce01" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42573 } }
I20260806 21:10:36.213326 14117 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae77f322ee1b468c8f709e0e4a0bce01 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:10:36.214799 14123 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae77f322ee1b468c8f709e0e4a0bce01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae77f322ee1b468c8f709e0e4a0bce01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae77f322ee1b468c8f709e0e4a0bce01" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42573 } } }
I20260806 21:10:36.214954 14123 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae77f322ee1b468c8f709e0e4a0bce01 [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:36.215385 14131 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:10:36.214812 14124 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae77f322ee1b468c8f709e0e4a0bce01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae77f322ee1b468c8f709e0e4a0bce01. Latest consensus state: current_term: 1 leader_uuid: "ae77f322ee1b468c8f709e0e4a0bce01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae77f322ee1b468c8f709e0e4a0bce01" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42573 } } }
I20260806 21:10:36.215917 14124 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae77f322ee1b468c8f709e0e4a0bce01 [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:36.218603 14131 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:10:36.224258 14131 catalog_manager.cc:1347] Generated new cluster ID: 762d3ba2fa13484a969b82dfe1d29d98
I20260806 21:10:36.224344 14131 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:10:36.278146 14131 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:10:36.279597 14131 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:10:36.285105 14131 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae77f322ee1b468c8f709e0e4a0bce01: Generated new TSK 0
I20260806 21:10:36.285974 14131 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:10:36.297621 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:42573
--builtin_ntp_servers=127.27.225.212:35231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 21:10:36.415045 14142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:36.415354 14142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:36.415429 14142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:36.419899 14142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:36.419978 14142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:36.420038 14142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:10:36.424957 14142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:35231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:42573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:36.426281 14142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:36.427843 14142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:36.438005 14150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:36.438007 14148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:36.438032 14147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:36.438649 14142 server_base.cc:1034] running on GCE node
I20260806 21:10:36.439119 14142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:36.439867 14142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:36.441102 14142 hybrid_clock.cc:648] HybridClock initialized: now 1786050636441067 us; error 48 us; skew 500 ppm
I20260806 21:10:36.443599 14142 webserver.cc:460] Webserver started at http://127.27.225.193:33753/ using document root <none> and password file <none>
I20260806 21:10:36.444244 14142 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:36.444350 14142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:36.444599 14142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:36.446849 14142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "3d24773c0b084237a6461725973ceae8"
format_stamp: "Formatted at 2026-08-06 21:10:36 on dist-test-slave-69rg"
server_key: "04fd1ac20a86a07f07deb4245eeed1ad"
server_key_iv: "3c0ef196f78e82359a689a8ed40f6862"
server_key_version: "encryptionkey@0"
I20260806 21:10:36.447448 14142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "3d24773c0b084237a6461725973ceae8"
format_stamp: "Formatted at 2026-08-06 21:10:36 on dist-test-slave-69rg"
server_key: "04fd1ac20a86a07f07deb4245eeed1ad"
server_key_iv: "3c0ef196f78e82359a689a8ed40f6862"
server_key_version: "encryptionkey@0"
I20260806 21:10:36.451332 14142 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:10:36.453902 14156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:36.455147 14142 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 21:10:36.455295 14142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "3d24773c0b084237a6461725973ceae8"
format_stamp: "Formatted at 2026-08-06 21:10:36 on dist-test-slave-69rg"
server_key: "04fd1ac20a86a07f07deb4245eeed1ad"
server_key_iv: "3c0ef196f78e82359a689a8ed40f6862"
server_key_version: "encryptionkey@0"
I20260806 21:10:36.455416 14142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:36.468735 14142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:36.469465 14142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:36.469676 14142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:36.470324 14142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:10:36.471303 14142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:10:36.471376 14142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:36.471449 14142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:10:36.471491 14142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:36.480876 14142 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:33585
I20260806 21:10:36.480894 14270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:33585 every 8 connection(s)
I20260806 21:10:36.481979 14142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:10:36.485769 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 14142
I20260806 21:10:36.485908 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:10:36.486300 28551 external_mini_cluster.cc:1428] Setting key 2ed730e820ac8a552df49e0e74c4fb87
I20260806 21:10:36.488503 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:42573
--builtin_ntp_servers=127.27.225.212:35231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 21:10:36.492712 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:36.483846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:42859 (local address 127.27.225.254:42573)
0806 21:10:36.483993 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:36.483997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:36.485017 (+ 1020us) server_negotiation.cc:407] Connection header received
0806 21:10:36.486854 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:36.486859 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:36.486921 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:36.487022 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:36.488136 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.488773 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.490566 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.490963 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.491082 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:36.492229 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:36.492232 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:36.492235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:36.492247 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:36.492266 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:36.492285 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:36.492360 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:36.492547 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 21:10:36.492598 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 21:10:36.494019 14273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:36.484233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:42573 (local address 127.27.225.193:42859)
0806 21:10:36.484867 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:36.484908 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:36.486568 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:36.487223 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:36.487234 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:36.487346 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:10:36.487977 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:36.487998 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.488909 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.488916 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:36.490421 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:36.490437 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.491126 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.491130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:36.491278 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:36.491931 (+ 653us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:10:36.491936 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:10:36.491990 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:10:36.492074 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:10:36.492388 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:10:36.492401 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:10:36.492410 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 21:10:36.492540 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 21:10:36.492872 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":112,"threads_started":1}
I20260806 21:10:36.495311 14271 heartbeater.cc:347] Connected to a master server at 127.27.225.254:42573
I20260806 21:10:36.495586 14271 heartbeater.cc:464] Registering TS with master...
I20260806 21:10:36.496281 14271 heartbeater.cc:511] Master 127.27.225.254:42573 requested a full tablet report, sending...
I20260806 21:10:36.497931 14081 ts_manager.cc:194] Registered new tserver with Master: 3d24773c0b084237a6461725973ceae8 (127.27.225.193:33585)
I20260806 21:10:36.499102 14081 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.225.193:42859
W20260806 21:10:36.605818 14274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:36.606241 14274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:36.606319 14274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:36.610581 14274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:36.610688 14274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:36.610788 14274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:10:36.616614 14274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:35231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:42573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:36.617831 14274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:36.619328 14274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:36.629189 14282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:36.629223 14279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:36.629189 14280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:36.629498 14274 server_base.cc:1034] running on GCE node
I20260806 21:10:36.629968 14274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:36.630643 14274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:36.631811 14274 hybrid_clock.cc:648] HybridClock initialized: now 1786050636631794 us; error 32 us; skew 500 ppm
I20260806 21:10:36.640848 14274 webserver.cc:460] Webserver started at http://127.27.225.194:35269/ using document root <none> and password file <none>
I20260806 21:10:36.641489 14274 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:36.641558 14274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:36.641790 14274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:36.644033 14274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "db6731a0fe91428d8b3ea4d1e5efdade"
format_stamp: "Formatted at 2026-08-06 21:10:36 on dist-test-slave-69rg"
server_key: "d8de6ba2f0ab23eac473114f84fae6bb"
server_key_iv: "c89b737767134e491e4c2f336fe89896"
server_key_version: "encryptionkey@0"
I20260806 21:10:36.644613 14274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "db6731a0fe91428d8b3ea4d1e5efdade"
format_stamp: "Formatted at 2026-08-06 21:10:36 on dist-test-slave-69rg"
server_key: "d8de6ba2f0ab23eac473114f84fae6bb"
server_key_iv: "c89b737767134e491e4c2f336fe89896"
server_key_version: "encryptionkey@0"
I20260806 21:10:36.648581 14274 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260806 21:10:36.651223 14288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:36.652480 14274 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 21:10:36.652612 14274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "db6731a0fe91428d8b3ea4d1e5efdade"
format_stamp: "Formatted at 2026-08-06 21:10:36 on dist-test-slave-69rg"
server_key: "d8de6ba2f0ab23eac473114f84fae6bb"
server_key_iv: "c89b737767134e491e4c2f336fe89896"
server_key_version: "encryptionkey@0"
I20260806 21:10:36.652734 14274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:36.680846 14274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:36.681655 14274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:36.681912 14274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:36.682585 14274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:10:36.683607 14274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:10:36.683683 14274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:36.683758 14274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:10:36.683804 14274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:36.693086 14274 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:40571
I20260806 21:10:36.693105 14408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:40571 every 8 connection(s)
I20260806 21:10:36.694124 14274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:10:36.697628 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 14274
I20260806 21:10:36.697767 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:10:36.698189 28551 external_mini_cluster.cc:1428] Setting key f2f44188da8109c0ee593b65aed0cc91
I20260806 21:10:36.700711 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:42573
--builtin_ntp_servers=127.27.225.212:35231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 21:10:36.705833 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:36.696095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:56635 (local address 127.27.225.254:42573)
0806 21:10:36.696248 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:36.696252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:36.697530 (+ 1278us) server_negotiation.cc:407] Connection header received
0806 21:10:36.699506 (+ 1976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:36.699509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:36.699555 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:36.699630 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:36.700900 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.701484 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.703285 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.703708 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.703843 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:36.705165 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:36.705171 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:36.705176 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:36.705196 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:36.705225 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:36.705239 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:36.705321 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:36.705486 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 21:10:36.705573 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 21:10:36.706661 14411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:36.696493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:42573 (local address 127.27.225.194:56635)
0806 21:10:36.697242 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:36.697366 (+ 124us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:36.699240 (+ 1874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:36.699836 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:36.699844 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:36.699935 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:10:36.700701 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:36.700725 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.701639 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.701646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:36.703132 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:36.703143 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.703894 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.703897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:36.704055 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:36.704834 (+ 779us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:10:36.704838 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:10:36.704900 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:10:36.704983 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:10:36.705327 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:10:36.705334 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:10:36.705338 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 21:10:36.705439 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 21:10:36.705689 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":124,"threads_started":1}
I20260806 21:10:36.707950 14409 heartbeater.cc:347] Connected to a master server at 127.27.225.254:42573
I20260806 21:10:36.708211 14409 heartbeater.cc:464] Registering TS with master...
I20260806 21:10:36.708760 14409 heartbeater.cc:511] Master 127.27.225.254:42573 requested a full tablet report, sending...
I20260806 21:10:36.709985 14081 ts_manager.cc:194] Registered new tserver with Master: db6731a0fe91428d8b3ea4d1e5efdade (127.27.225.194:40571)
I20260806 21:10:36.710889 14081 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.225.194:56635
W20260806 21:10:36.817889 14412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:36.818193 14412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:36.818233 14412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:36.822216 14412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:36.822274 14412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:36.822327 14412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:10:36.827088 14412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:35231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:42573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:36.828080 14412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:36.829361 14412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:36.838887 14420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:36.838892 14417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:36.838892 14418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:36.839318 14412 server_base.cc:1034] running on GCE node
I20260806 21:10:36.839821 14412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:36.840420 14412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:36.841629 14412 hybrid_clock.cc:648] HybridClock initialized: now 1786050636841614 us; error 25 us; skew 500 ppm
I20260806 21:10:36.843690 14412 webserver.cc:460] Webserver started at http://127.27.225.195:40745/ using document root <none> and password file <none>
I20260806 21:10:36.844298 14412 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:36.844398 14412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:36.844622 14412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:36.846902 14412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "690f6973bf3847b4bdb987a6bb4c7619"
format_stamp: "Formatted at 2026-08-06 21:10:36 on dist-test-slave-69rg"
server_key: "bb11ba9a0f3ab74c87e5788c2249875b"
server_key_iv: "e006fc740f55014a101393a2a394121a"
server_key_version: "encryptionkey@0"
I20260806 21:10:36.847714 14412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "690f6973bf3847b4bdb987a6bb4c7619"
format_stamp: "Formatted at 2026-08-06 21:10:36 on dist-test-slave-69rg"
server_key: "bb11ba9a0f3ab74c87e5788c2249875b"
server_key_iv: "e006fc740f55014a101393a2a394121a"
server_key_version: "encryptionkey@0"
I20260806 21:10:36.852120 14412 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:10:36.854945 14426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:36.856339 14412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:10:36.856491 14412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "690f6973bf3847b4bdb987a6bb4c7619"
format_stamp: "Formatted at 2026-08-06 21:10:36 on dist-test-slave-69rg"
server_key: "bb11ba9a0f3ab74c87e5788c2249875b"
server_key_iv: "e006fc740f55014a101393a2a394121a"
server_key_version: "encryptionkey@0"
I20260806 21:10:36.856657 14412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:36.877703 14412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:36.878628 14412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:36.878868 14412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:36.879526 14412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:10:36.880519 14412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:10:36.880596 14412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:36.880677 14412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:10:36.880722 14412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:36.890355 14412 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:34653
I20260806 21:10:36.890380 14542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:34653 every 8 connection(s)
I20260806 21:10:36.891314 14412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:10:36.899664 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 14412
I20260806 21:10:36.899794 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:10:36.900113 28551 external_mini_cluster.cc:1428] Setting key 913b90b025109d66adcf52a60863ad71
I20260806 21:10:36.901507 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:36.893303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:33141 (local address 127.27.225.254:42573)
0806 21:10:36.893466 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:36.893470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:36.894632 (+ 1162us) server_negotiation.cc:407] Connection header received
0806 21:10:36.896212 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:36.896216 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:36.896271 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:36.896363 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:36.897447 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.898083 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.899334 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.899560 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.899647 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:36.900976 (+ 1329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:36.900980 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:36.900983 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:36.901004 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:36.901025 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:36.901034 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:36.901095 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:36.901291 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 21:10:36.901362 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 21:10:36.902571 14545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:36.893685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:42573 (local address 127.27.225.195:33141)
0806 21:10:36.894455 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:36.894502 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:36.895939 (+ 1437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:36.896574 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:36.896582 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:36.896681 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:10:36.897262 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:36.897276 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.898250 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.898253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:36.899200 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:36.899211 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.899748 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.899753 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:36.899974 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:36.900714 (+ 740us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:10:36.900716 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:10:36.900761 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:10:36.900830 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:10:36.901122 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:10:36.901132 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:10:36.901139 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 21:10:36.901289 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 21:10:36.901598 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":144,"threads_started":1}
I20260806 21:10:36.903792 14543 heartbeater.cc:347] Connected to a master server at 127.27.225.254:42573
I20260806 21:10:36.904073 14543 heartbeater.cc:464] Registering TS with master...
I20260806 21:10:36.904697 14543 heartbeater.cc:511] Master 127.27.225.254:42573 requested a full tablet report, sending...
I20260806 21:10:36.905789 14081 ts_manager.cc:194] Registered new tserver with Master: 690f6973bf3847b4bdb987a6bb4c7619 (127.27.225.195:34653)
I20260806 21:10:36.906487 14081 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.225.195:33141
I20260806 21:10:36.915132 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:10:36.921489 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:36.917358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48000 (local address 127.27.225.254:42573)
0806 21:10:36.917534 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:36.917540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:36.917685 (+ 145us) server_negotiation.cc:407] Connection header received
0806 21:10:36.917883 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:36.917886 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:36.917938 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:36.918038 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:36.918387 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.918922 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.919816 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.920037 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.920146 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:36.921023 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:36.921027 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:36.921029 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:36.921041 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:36.921061 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:36.921069 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:36.921165 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:36.921259 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 21:10:36.921327 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 21:10:36.924546 14081 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48000:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:10:36.928007 14081 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:10:36.943035 14557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:36.937596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59938 (local address 127.27.225.195:34653)
0806 21:10:36.937941 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:36.937954 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:36.938135 (+ 181us) server_negotiation.cc:407] Connection header received
0806 21:10:36.938249 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:36.938255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:36.938477 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:36.938636 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:36.938990 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.940041 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.941677 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.942314 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.942456 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:36.942643 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:36.942742 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 21:10:36.942841 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":132,"threads_started":1}
I20260806 21:10:36.943027 14555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:36.936996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:34653 (local address 127.0.0.1:59938)
0806 21:10:36.938004 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:36.938019 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:36.938129 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:36.938743 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:36.938745 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:36.938757 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:36.938882 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:36.938890 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.940154 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.940157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:36.941542 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:36.941552 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.942445 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.942449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:36.942555 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:36.942591 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 21:10:36.942714 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 21:10:36.942845 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":941,"mutex_wait_us":54,"thread_start_us":113,"threads_started":1}
I20260806 21:10:36.943038 14554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:36.936849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:40571 (local address 127.0.0.1:36116)
0806 21:10:36.937511 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:36.937571 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:36.937723 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:36.938663 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:36.938670 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:36.938697 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:36.939053 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:36.939060 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.940279 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.940283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:36.941552 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:36.941562 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.942644 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.942648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:36.942736 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:36.942761 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 21:10:36.942810 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 21:10:36.942877 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":223,"thread_start_us":88,"threads_started":1}
I20260806 21:10:36.943593 14556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:36.937000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36116 (local address 127.27.225.194:40571)
0806 21:10:36.938140 (+ 1140us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:36.938153 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:36.938180 (+ 27us) server_negotiation.cc:407] Connection header received
0806 21:10:36.938258 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:36.938263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:36.938476 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:36.938885 (+ 409us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:36.939178 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.940158 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.941711 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.942488 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.943125 (+ 637us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:36.943280 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:36.943330 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 21:10:36.943421 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1002,"thread_start_us":176,"threads_started":1}
I20260806 21:10:36.944205 14558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:36.938497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:33585 (local address 127.0.0.1:60634)
0806 21:10:36.939260 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:36.939273 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:36.939362 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:36.939774 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:36.939777 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:36.939795 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:36.939873 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:36.939878 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.941273 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.941276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:36.942486 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:36.942497 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.943805 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.943811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:36.943907 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:36.943929 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 21:10:36.944002 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 21:10:36.944066 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"mutex_wait_us":31,"thread_start_us":88,"threads_started":1}
I20260806 21:10:36.944742 14559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:36.938662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60634 (local address 127.27.225.193:33585)
0806 21:10:36.938944 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:36.938957 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:36.939376 (+ 419us) server_negotiation.cc:407] Connection header received
0806 21:10:36.939456 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:36.939461 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:36.939628 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:36.939771 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:36.939975 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.941152 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.942627 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:36.943409 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:36.943533 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:36.943731 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:36.944470 (+ 739us) server_negotiation.cc:299] Negotiation successful
0806 21:10:36.944584 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":107,"threads_started":1}
I20260806 21:10:36.946580 14338 tablet_service.cc:1467] Processing CreateTablet for tablet d5a9716155584ec786fbe471db338dc4 (DEFAULT_TABLE table=test-table [id=3c681bdda3ad4756912a8d57e32fe2fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:10:36.946630 14475 tablet_service.cc:1467] Processing CreateTablet for tablet d5a9716155584ec786fbe471db338dc4 (DEFAULT_TABLE table=test-table [id=3c681bdda3ad4756912a8d57e32fe2fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:10:36.947611 14338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5a9716155584ec786fbe471db338dc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:36.947693 14475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5a9716155584ec786fbe471db338dc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:36.947328 14204 tablet_service.cc:1467] Processing CreateTablet for tablet d5a9716155584ec786fbe471db338dc4 (DEFAULT_TABLE table=test-table [id=3c681bdda3ad4756912a8d57e32fe2fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:10:36.948239 14204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5a9716155584ec786fbe471db338dc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:36.953855 14560 tablet_bootstrap.cc:492] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade: Bootstrap starting.
I20260806 21:10:36.955366 14562 tablet_bootstrap.cc:492] T d5a9716155584ec786fbe471db338dc4 P 690f6973bf3847b4bdb987a6bb4c7619: Bootstrap starting.
I20260806 21:10:36.956238 14560 tablet_bootstrap.cc:654] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:36.957104 14560 log.cc:824] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:36.957568 14562 tablet_bootstrap.cc:654] T d5a9716155584ec786fbe471db338dc4 P 690f6973bf3847b4bdb987a6bb4c7619: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:36.957882 14561 tablet_bootstrap.cc:492] T d5a9716155584ec786fbe471db338dc4 P 3d24773c0b084237a6461725973ceae8: Bootstrap starting.
I20260806 21:10:36.958564 14562 log.cc:824] T d5a9716155584ec786fbe471db338dc4 P 690f6973bf3847b4bdb987a6bb4c7619: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:36.959537 14560 tablet_bootstrap.cc:492] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade: No bootstrap required, opened a new log
I20260806 21:10:36.959756 14560 ts_tablet_manager.cc:1397] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 21:10:36.960181 14561 tablet_bootstrap.cc:654] T d5a9716155584ec786fbe471db338dc4 P 3d24773c0b084237a6461725973ceae8: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:36.960840 14562 tablet_bootstrap.cc:492] T d5a9716155584ec786fbe471db338dc4 P 690f6973bf3847b4bdb987a6bb4c7619: No bootstrap required, opened a new log
I20260806 21:10:36.960958 14561 log.cc:824] T d5a9716155584ec786fbe471db338dc4 P 3d24773c0b084237a6461725973ceae8: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:36.961012 14562 ts_tablet_manager.cc:1397] T d5a9716155584ec786fbe471db338dc4 P 690f6973bf3847b4bdb987a6bb4c7619: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260806 21:10:36.962944 14561 tablet_bootstrap.cc:492] T d5a9716155584ec786fbe471db338dc4 P 3d24773c0b084237a6461725973ceae8: No bootstrap required, opened a new log
I20260806 21:10:36.963135 14561 ts_tablet_manager.cc:1397] T d5a9716155584ec786fbe471db338dc4 P 3d24773c0b084237a6461725973ceae8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 21:10:36.963783 14560 raft_consensus.cc:348] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db6731a0fe91428d8b3ea4d1e5efdade" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 40571 } } peers { permanent_uuid: "690f6973bf3847b4bdb987a6bb4c7619" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34653 } } peers { permanent_uuid: "3d24773c0b084237a6461725973ceae8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 33585 } }
I20260806 21:10:36.964023 14560 raft_consensus.cc:374] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:36.964075 14560 raft_consensus.cc:729] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db6731a0fe91428d8b3ea4d1e5efdade, State: Initialized, Role: FOLLOWER
I20260806 21:10:36.964524 14560 consensus_queue.cc:260] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db6731a0fe91428d8b3ea4d1e5efdade" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 40571 } } peers { permanent_uuid: "690f6973bf3847b4bdb987a6bb4c7619" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34653 } } peers { permanent_uuid: "3d24773c0b084237a6461725973ceae8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 33585 } }
I20260806 21:10:36.965278 14562 raft_consensus.cc:348] T d5a9716155584ec786fbe471db338dc4 P 690f6973bf3847b4bdb987a6bb4c7619 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db6731a0fe91428d8b3ea4d1e5efdade" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 40571 } } peers { permanent_uuid: "690f6973bf3847b4bdb987a6bb4c7619" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34653 } } peers { permanent_uuid: "3d24773c0b084237a6461725973ceae8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 33585 } }
I20260806 21:10:36.965520 14562 raft_consensus.cc:374] T d5a9716155584ec786fbe471db338dc4 P 690f6973bf3847b4bdb987a6bb4c7619 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:36.965567 14562 raft_consensus.cc:729] T d5a9716155584ec786fbe471db338dc4 P 690f6973bf3847b4bdb987a6bb4c7619 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 690f6973bf3847b4bdb987a6bb4c7619, State: Initialized, Role: FOLLOWER
I20260806 21:10:36.965584 14560 ts_tablet_manager.cc:1428] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 21:10:36.965953 14409 heartbeater.cc:503] Master 127.27.225.254:42573 was elected leader, sending a full tablet report...
I20260806 21:10:36.966043 14562 consensus_queue.cc:260] T d5a9716155584ec786fbe471db338dc4 P 690f6973bf3847b4bdb987a6bb4c7619 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db6731a0fe91428d8b3ea4d1e5efdade" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 40571 } } peers { permanent_uuid: "690f6973bf3847b4bdb987a6bb4c7619" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34653 } } peers { permanent_uuid: "3d24773c0b084237a6461725973ceae8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 33585 } }
I20260806 21:10:36.966727 14543 heartbeater.cc:503] Master 127.27.225.254:42573 was elected leader, sending a full tablet report...
I20260806 21:10:36.967583 14561 raft_consensus.cc:348] T d5a9716155584ec786fbe471db338dc4 P 3d24773c0b084237a6461725973ceae8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db6731a0fe91428d8b3ea4d1e5efdade" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 40571 } } peers { permanent_uuid: "690f6973bf3847b4bdb987a6bb4c7619" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34653 } } peers { permanent_uuid: "3d24773c0b084237a6461725973ceae8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 33585 } }
I20260806 21:10:36.967825 14561 raft_consensus.cc:374] T d5a9716155584ec786fbe471db338dc4 P 3d24773c0b084237a6461725973ceae8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:36.967883 14561 raft_consensus.cc:729] T d5a9716155584ec786fbe471db338dc4 P 3d24773c0b084237a6461725973ceae8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d24773c0b084237a6461725973ceae8, State: Initialized, Role: FOLLOWER
I20260806 21:10:36.967968 14562 ts_tablet_manager.cc:1428] T d5a9716155584ec786fbe471db338dc4 P 690f6973bf3847b4bdb987a6bb4c7619: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 21:10:36.968302 14561 consensus_queue.cc:260] T d5a9716155584ec786fbe471db338dc4 P 3d24773c0b084237a6461725973ceae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db6731a0fe91428d8b3ea4d1e5efdade" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 40571 } } peers { permanent_uuid: "690f6973bf3847b4bdb987a6bb4c7619" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34653 } } peers { permanent_uuid: "3d24773c0b084237a6461725973ceae8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 33585 } }
I20260806 21:10:36.968935 14271 heartbeater.cc:503] Master 127.27.225.254:42573 was elected leader, sending a full tablet report...
I20260806 21:10:36.969233 14561 ts_tablet_manager.cc:1428] T d5a9716155584ec786fbe471db338dc4 P 3d24773c0b084237a6461725973ceae8: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
W20260806 21:10:36.983197 14272 tablet.cc:2369] T d5a9716155584ec786fbe471db338dc4 P 3d24773c0b084237a6461725973ceae8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:10:37.054369 14566 raft_consensus.cc:482] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:10:37.054633 14566 raft_consensus.cc:504] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db6731a0fe91428d8b3ea4d1e5efdade" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 40571 } } peers { permanent_uuid: "690f6973bf3847b4bdb987a6bb4c7619" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34653 } } peers { permanent_uuid: "3d24773c0b084237a6461725973ceae8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 33585 } }
I20260806 21:10:37.055891 14566 leader_election.cc:288] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 690f6973bf3847b4bdb987a6bb4c7619 (127.27.225.195:34653), 3d24773c0b084237a6461725973ceae8 (127.27.225.193:33585)
I20260806 21:10:37.061084 14557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:37.056425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:59111 (local address 127.27.225.195:34653)
0806 21:10:37.056574 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:37.056579 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:37.056596 (+ 17us) server_negotiation.cc:407] Connection header received
0806 21:10:37.056675 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:37.056677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:37.056720 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:37.056789 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:37.057200 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:37.057954 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:37.059586 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:37.060225 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:37.060355 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:37.060421 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:37.060880 (+ 459us) server_negotiation.cc:299] Negotiation successful
0806 21:10:37.060962 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 21:10:37.061116 14411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:37.056214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:34653 (local address 127.27.225.194:59111)
0806 21:10:37.056381 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:37.056400 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:37.056525 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:37.056834 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:37.056837 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:37.056866 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:37.057038 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:37.057045 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:37.058114 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:37.058118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:37.059452 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:37.059463 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:37.060418 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:37.060425 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:37.060663 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:37.060719 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 21:10:37.060798 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 21:10:37.060955 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":184,"spinlock_wait_cycles":2944}
I20260806 21:10:37.061084 14559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:37.056356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:41991 (local address 127.27.225.193:33585)
0806 21:10:37.056505 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:37.056510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:37.056878 (+ 368us) server_negotiation.cc:407] Connection header received
0806 21:10:37.056974 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:37.056979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:37.057030 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:37.057107 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:37.057416 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:37.058235 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:37.059672 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:37.060330 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:37.060429 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:37.060482 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:37.060860 (+ 378us) server_negotiation.cc:299] Negotiation successful
0806 21:10:37.060949 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 21:10:37.061123 14569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:37.056214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:33585 (local address 127.27.225.194:41991)
0806 21:10:37.056739 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:37.056751 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:37.056874 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:37.057139 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:37.057143 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:37.057166 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:37.057301 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:37.057307 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:37.058349 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:37.058352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:37.059502 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:37.059513 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:37.060444 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:37.060447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:37.060667 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:37.060716 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 21:10:37.060798 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 21:10:37.060940 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":258,"spinlock_wait_cycles":768,"thread_start_us":118,"threads_started":1}
I20260806 21:10:37.062142 14224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5a9716155584ec786fbe471db338dc4" candidate_uuid: "db6731a0fe91428d8b3ea4d1e5efdade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d24773c0b084237a6461725973ceae8" is_pre_election: true
I20260806 21:10:37.062144 14496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5a9716155584ec786fbe471db338dc4" candidate_uuid: "db6731a0fe91428d8b3ea4d1e5efdade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "690f6973bf3847b4bdb987a6bb4c7619" is_pre_election: true
I20260806 21:10:37.062482 14224 raft_consensus.cc:2454] T d5a9716155584ec786fbe471db338dc4 P 3d24773c0b084237a6461725973ceae8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db6731a0fe91428d8b3ea4d1e5efdade in term 0.
I20260806 21:10:37.062482 14496 raft_consensus.cc:2454] T d5a9716155584ec786fbe471db338dc4 P 690f6973bf3847b4bdb987a6bb4c7619 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db6731a0fe91428d8b3ea4d1e5efdade in term 0.
I20260806 21:10:37.063043 14293 leader_election.cc:302] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3d24773c0b084237a6461725973ceae8, 690f6973bf3847b4bdb987a6bb4c7619, db6731a0fe91428d8b3ea4d1e5efdade; no voters:
I20260806 21:10:37.063369 14566 raft_consensus.cc:2781] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:10:37.063477 14566 raft_consensus.cc:482] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:10:37.063541 14566 raft_consensus.cc:3037] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:37.064661 14566 raft_consensus.cc:504] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db6731a0fe91428d8b3ea4d1e5efdade" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 40571 } } peers { permanent_uuid: "690f6973bf3847b4bdb987a6bb4c7619" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34653 } } peers { permanent_uuid: "3d24773c0b084237a6461725973ceae8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 33585 } }
I20260806 21:10:37.065028 14566 leader_election.cc:288] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade [CANDIDATE]: Term 1 election: Requested vote from peers 690f6973bf3847b4bdb987a6bb4c7619 (127.27.225.195:34653), 3d24773c0b084237a6461725973ceae8 (127.27.225.193:33585)
I20260806 21:10:37.065444 14224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5a9716155584ec786fbe471db338dc4" candidate_uuid: "db6731a0fe91428d8b3ea4d1e5efdade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d24773c0b084237a6461725973ceae8"
I20260806 21:10:37.065526 14496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5a9716155584ec786fbe471db338dc4" candidate_uuid: "db6731a0fe91428d8b3ea4d1e5efdade" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "690f6973bf3847b4bdb987a6bb4c7619"
I20260806 21:10:37.065594 14224 raft_consensus.cc:3037] T d5a9716155584ec786fbe471db338dc4 P 3d24773c0b084237a6461725973ceae8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:37.065640 14496 raft_consensus.cc:3037] T d5a9716155584ec786fbe471db338dc4 P 690f6973bf3847b4bdb987a6bb4c7619 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:37.066982 14496 raft_consensus.cc:2454] T d5a9716155584ec786fbe471db338dc4 P 690f6973bf3847b4bdb987a6bb4c7619 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db6731a0fe91428d8b3ea4d1e5efdade in term 1.
I20260806 21:10:37.066982 14224 raft_consensus.cc:2454] T d5a9716155584ec786fbe471db338dc4 P 3d24773c0b084237a6461725973ceae8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db6731a0fe91428d8b3ea4d1e5efdade in term 1.
I20260806 21:10:37.067348 14293 leader_election.cc:302] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d24773c0b084237a6461725973ceae8, db6731a0fe91428d8b3ea4d1e5efdade; no voters:
I20260806 21:10:37.067553 14566 raft_consensus.cc:2781] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:10:37.067836 14566 raft_consensus.cc:686] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade [term 1 LEADER]: Becoming Leader. State: Replica: db6731a0fe91428d8b3ea4d1e5efdade, State: Running, Role: LEADER
I20260806 21:10:37.068195 14566 consensus_queue.cc:237] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db6731a0fe91428d8b3ea4d1e5efdade" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 40571 } } peers { permanent_uuid: "690f6973bf3847b4bdb987a6bb4c7619" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34653 } } peers { permanent_uuid: "3d24773c0b084237a6461725973ceae8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 33585 } }
I20260806 21:10:37.071300 14080 catalog_manager.cc:5495] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade reported cstate change: term changed from 0 to 1, leader changed from <none> to db6731a0fe91428d8b3ea4d1e5efdade (127.27.225.194). New cstate: current_term: 1 leader_uuid: "db6731a0fe91428d8b3ea4d1e5efdade" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db6731a0fe91428d8b3ea4d1e5efdade" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 40571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "690f6973bf3847b4bdb987a6bb4c7619" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d24773c0b084237a6461725973ceae8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 33585 } health_report { overall_health: UNKNOWN } } }
I20260806 21:10:37.096591 14556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:37.092858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36128 (local address 127.27.225.194:40571)
0806 21:10:37.092988 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:37.092992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:37.093006 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:10:37.093114 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:37.093116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:37.093174 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:37.093275 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:10:37.093601 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:37.094283 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:37.095121 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:37.095379 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:37.095470 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:37.095761 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:10:37.096242 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:10:37.096344 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:37.096419 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 21:10:37.096471 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 21:10:37.103016 14496 raft_consensus.cc:1261] T d5a9716155584ec786fbe471db338dc4 P 690f6973bf3847b4bdb987a6bb4c7619 [term 1 FOLLOWER]: Refusing update from remote peer db6731a0fe91428d8b3ea4d1e5efdade: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:10:37.103016 14224 raft_consensus.cc:1261] T d5a9716155584ec786fbe471db338dc4 P 3d24773c0b084237a6461725973ceae8 [term 1 FOLLOWER]: Refusing update from remote peer db6731a0fe91428d8b3ea4d1e5efdade: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:10:37.103893 14566 consensus_queue.cc:1035] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade [LEADER]: Connected to new peer: Peer: permanent_uuid: "690f6973bf3847b4bdb987a6bb4c7619" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:10:37.104197 14570 consensus_queue.cc:1035] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d24773c0b084237a6461725973ceae8" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 33585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:10:37.115674 14575 mvcc.cc:205] Tried to move back new op lower bound from 7315663409565335552 to 7315663409434107904. Current Snapshot: MvccSnapshot[applied={T|T < 7315663409565335552}]
I20260806 21:10:37.115903 14574 mvcc.cc:205] Tried to move back new op lower bound from 7315663409565335552 to 7315663409434107904. Current Snapshot: MvccSnapshot[applied={T|T < 7315663409565335552 or (T in {7315663409565335552})}]
I20260806 21:10:37.120071 14081 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:10:37.120262 14081 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:10:37.122571 14081 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ae77f322ee1b468c8f709e0e4a0bce01: Sending DeleteTablet for 3 replicas of tablet d5a9716155584ec786fbe471db338dc4
I20260806 21:10:37.123389 14204 tablet_service.cc:1514] Processing DeleteTablet for tablet d5a9716155584ec786fbe471db338dc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:10:37 UTC) from {username='slave'} at 127.0.0.1:60634
I20260806 21:10:37.123458 14475 tablet_service.cc:1514] Processing DeleteTablet for tablet d5a9716155584ec786fbe471db338dc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:10:37 UTC) from {username='slave'} at 127.0.0.1:59938
I20260806 21:10:37.123939 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 14142
I20260806 21:10:37.124030 14338 tablet_service.cc:1514] Processing DeleteTablet for tablet d5a9716155584ec786fbe471db338dc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:10:37 UTC) from {username='slave'} at 127.0.0.1:36116
I20260806 21:10:37.124706 14584 tablet_replica.cc:330] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade: stopping tablet replica
I20260806 21:10:37.125032 14584 raft_consensus.cc:2229] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:10:37.125382 14584 raft_consensus.cc:2258] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:10:37.127074 14584 ts_tablet_manager.cc:1905] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:10:37.129570 14584 ts_tablet_manager.cc:1918] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:10:37.129653 14584 log.cc:1196] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-1/wal/wals/d5a9716155584ec786fbe471db338dc4
I20260806 21:10:37.130054 14584 ts_tablet_manager.cc:1939] T d5a9716155584ec786fbe471db338dc4 P db6731a0fe91428d8b3ea4d1e5efdade: Deleting consensus metadata
I20260806 21:10:37.131276 14068 catalog_manager.cc:4841] TS db6731a0fe91428d8b3ea4d1e5efdade (127.27.225.194:40571): tablet d5a9716155584ec786fbe471db338dc4 (table test-table [id=3c681bdda3ad4756912a8d57e32fe2fa]) successfully deleted
I20260806 21:10:37.131487 14583 tablet_replica.cc:330] T d5a9716155584ec786fbe471db338dc4 P 690f6973bf3847b4bdb987a6bb4c7619: stopping tablet replica
I20260806 21:10:37.131798 14583 raft_consensus.cc:2229] T d5a9716155584ec786fbe471db338dc4 P 690f6973bf3847b4bdb987a6bb4c7619 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:10:37.132115 14583 raft_consensus.cc:2258] T d5a9716155584ec786fbe471db338dc4 P 690f6973bf3847b4bdb987a6bb4c7619 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 21:10:37.133014 14068 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.225.193:33585 (error 108)
I20260806 21:10:37.133119 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 14274
I20260806 21:10:37.134017 14583 ts_tablet_manager.cc:1905] T d5a9716155584ec786fbe471db338dc4 P 690f6973bf3847b4bdb987a6bb4c7619: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:10:37.136662 14583 ts_tablet_manager.cc:1918] T d5a9716155584ec786fbe471db338dc4 P 690f6973bf3847b4bdb987a6bb4c7619: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:10:37.136785 14583 log.cc:1196] T d5a9716155584ec786fbe471db338dc4 P 690f6973bf3847b4bdb987a6bb4c7619: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/d5a9716155584ec786fbe471db338dc4
I20260806 21:10:37.137166 14583 ts_tablet_manager.cc:1939] T d5a9716155584ec786fbe471db338dc4 P 690f6973bf3847b4bdb987a6bb4c7619: Deleting consensus metadata
I20260806 21:10:37.138492 14065 catalog_manager.cc:4841] TS 690f6973bf3847b4bdb987a6bb4c7619 (127.27.225.195:34653): tablet d5a9716155584ec786fbe471db338dc4 (table test-table [id=3c681bdda3ad4756912a8d57e32fe2fa]) successfully deleted
I20260806 21:10:37.139220 14558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:37.138898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:33585 (local address 127.0.0.1:60644)
0806 21:10:37.139032 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:37.139097 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.193:33585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 21:10:37.139390 14068 catalog_manager.cc:4584] TS 3d24773c0b084237a6461725973ceae8 (127.27.225.193:33585): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d5a9716155584ec786fbe471db338dc4: Network error: Client connection negotiation failed: client connection to 127.27.225.193:33585: connect: Connection refused (error 111)
I20260806 21:10:37.140129 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 14412
I20260806 21:10:37.146868 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 14049
2026-08-06T21:10:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1182 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:10:37 dist-test-slave-69rg krb5kdc[14588](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:10:37 dist-test-slave-69rg krb5kdc[14588](info): set up 2 sockets
Aug 06 21:10:37 dist-test-slave-69rg krb5kdc[14588](info): commencing operation
krb5kdc: starting...
W20260806 21:10:38.241501 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:10:38 dist-test-slave-69rg krb5kdc[14588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:10:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:10:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:10:38.381726 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:36421
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:44529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:36421
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:10:38.494325 14602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:38.494637 14602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:38.494730 14602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:38.498737 14602 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:10:38.498836 14602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:38.498875 14602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:38.498912 14602 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:10:38.498947 14602 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:10:38.504248 14602 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:44529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:36421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:36421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:38.505409 14602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:38.506510 14602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:38.515741 14608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:38.515743 14610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:38.515736 14607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:38.516119 14602 server_base.cc:1034] running on GCE node
I20260806 21:10:38.516706 14602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:38.517940 14602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:38.519116 14602 hybrid_clock.cc:648] HybridClock initialized: now 1786050638519088 us; error 42 us; skew 500 ppm
Aug 06 21:10:38 dist-test-slave-69rg krb5kdc[14588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050638, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:38.522346 14602 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:10:38.523305 14602 webserver.cc:460] Webserver started at http://127.27.225.254:33937/ using document root <none> and password file <none>
I20260806 21:10:38.523931 14602 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:38.524032 14602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:38.524281 14602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:38.526594 14602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "3c36b72c72094cfa8f01546b15063e6f"
format_stamp: "Formatted at 2026-08-06 21:10:38 on dist-test-slave-69rg"
server_key: "65fb6a5af6a603b238d774f932dd480a"
server_key_iv: "00f9afaa45a7fecc318490efbfaea91d"
server_key_version: "encryptionkey@0"
I20260806 21:10:38.527237 14602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "3c36b72c72094cfa8f01546b15063e6f"
format_stamp: "Formatted at 2026-08-06 21:10:38 on dist-test-slave-69rg"
server_key: "65fb6a5af6a603b238d774f932dd480a"
server_key_iv: "00f9afaa45a7fecc318490efbfaea91d"
server_key_version: "encryptionkey@0"
I20260806 21:10:38.531575 14602 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:10:38.534341 14617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:38.535609 14602 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:10:38.535782 14602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "3c36b72c72094cfa8f01546b15063e6f"
format_stamp: "Formatted at 2026-08-06 21:10:38 on dist-test-slave-69rg"
server_key: "65fb6a5af6a603b238d774f932dd480a"
server_key_iv: "00f9afaa45a7fecc318490efbfaea91d"
server_key_version: "encryptionkey@0"
I20260806 21:10:38.535912 14602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:38.552349 14602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:38.556354 14602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:38.556582 14602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:38.564294 14602 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:36421
I20260806 21:10:38.564325 14669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:36421 every 8 connection(s)
I20260806 21:10:38.565308 14602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:10:38.568132 14670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:38.569205 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 14602
I20260806 21:10:38.569370 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:10:38.569715 28551 external_mini_cluster.cc:1428] Setting key 4fd14070dc8c299812fd5ed318f76220
I20260806 21:10:38.574333 14670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c36b72c72094cfa8f01546b15063e6f: Bootstrap starting.
I20260806 21:10:38.576563 14670 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c36b72c72094cfa8f01546b15063e6f: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:38.577322 14670 log.cc:824] T 00000000000000000000000000000000 P 3c36b72c72094cfa8f01546b15063e6f: Log is configured to *not* fsync() on all Append() calls
Aug 06 21:10:38 dist-test-slave-69rg krb5kdc[14588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:10:38.579497 14670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c36b72c72094cfa8f01546b15063e6f: No bootstrap required, opened a new log
I20260806 21:10:38.582991 14670 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c36b72c72094cfa8f01546b15063e6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c36b72c72094cfa8f01546b15063e6f" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 36421 } }
I20260806 21:10:38.583259 14670 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c36b72c72094cfa8f01546b15063e6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:38.583361 14670 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c36b72c72094cfa8f01546b15063e6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c36b72c72094cfa8f01546b15063e6f, State: Initialized, Role: FOLLOWER
I20260806 21:10:38.583856 14670 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c36b72c72094cfa8f01546b15063e6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c36b72c72094cfa8f01546b15063e6f" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 36421 } }
I20260806 21:10:38.584012 14670 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c36b72c72094cfa8f01546b15063e6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:10:38.584101 14670 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c36b72c72094cfa8f01546b15063e6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:10:38.584203 14670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c36b72c72094cfa8f01546b15063e6f [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:38.584378 14673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:38.571230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53108 (local address 127.27.225.254:36421)
0806 21:10:38.571768 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:38.571784 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:38.571821 (+ 37us) server_negotiation.cc:407] Connection header received
0806 21:10:38.572521 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:38.572538 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:38.572866 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:38.573217 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:38.574112 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:38.575140 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:38.576008 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:38.576475 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:38.576647 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:38.579562 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:38.579591 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:38.579612 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:10:38.579641 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:38.581384 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:38.581864 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:38.581870 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:38.581876 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:38.581995 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:38.582293 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:38.582298 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:38.582300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:38.582718 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:38.582933 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:38.583175 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 21:10:38.583500 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":168,"threads_started":1}
I20260806 21:10:38.585421 14670 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c36b72c72094cfa8f01546b15063e6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c36b72c72094cfa8f01546b15063e6f" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 36421 } }
I20260806 21:10:38.585753 14670 leader_election.cc:302] T 00000000000000000000000000000000 P 3c36b72c72094cfa8f01546b15063e6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c36b72c72094cfa8f01546b15063e6f; no voters:
I20260806 21:10:38.586123 14670 leader_election.cc:288] T 00000000000000000000000000000000 P 3c36b72c72094cfa8f01546b15063e6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:10:38.586205 14675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c36b72c72094cfa8f01546b15063e6f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:10:38.586433 14675 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c36b72c72094cfa8f01546b15063e6f [term 1 LEADER]: Becoming Leader. State: Replica: 3c36b72c72094cfa8f01546b15063e6f, State: Running, Role: LEADER
I20260806 21:10:38.586786 14675 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c36b72c72094cfa8f01546b15063e6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c36b72c72094cfa8f01546b15063e6f" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 36421 } }
I20260806 21:10:38.587448 14670 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c36b72c72094cfa8f01546b15063e6f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:10:38.588058 14675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c36b72c72094cfa8f01546b15063e6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c36b72c72094cfa8f01546b15063e6f. Latest consensus state: current_term: 1 leader_uuid: "3c36b72c72094cfa8f01546b15063e6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c36b72c72094cfa8f01546b15063e6f" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 36421 } } }
I20260806 21:10:38.588284 14675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c36b72c72094cfa8f01546b15063e6f [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:38.588514 14676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c36b72c72094cfa8f01546b15063e6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c36b72c72094cfa8f01546b15063e6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c36b72c72094cfa8f01546b15063e6f" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 36421 } } }
I20260806 21:10:38.588706 14676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c36b72c72094cfa8f01546b15063e6f [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:38.588759 14684 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:10:38.592145 14684 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:10:38.598455 14684 catalog_manager.cc:1347] Generated new cluster ID: 92100e008aac4d02a43c9bedd200a11d
I20260806 21:10:38.598604 14684 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:10:38.612665 14684 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:10:38.614060 14684 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:10:38.629221 14684 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c36b72c72094cfa8f01546b15063e6f: Generated new TSK 0
I20260806 21:10:38.630105 14684 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.225.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
I20260806 21:10:38.665581 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:36421
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:44529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:10:38.784524 14697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:38.784855 14697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:38.784935 14697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:38.789302 14697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:38.789408 14697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:38.789502 14697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:10:38.795315 14697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:44529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:36421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:38.796478 14697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:38.797533 14697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:38.807631 14705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:38.807682 14697 server_base.cc:1034] running on GCE node
W20260806 21:10:38.807782 14703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:38.807687 14702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:38.809065 14697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:38.809759 14697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:38.810984 14697 hybrid_clock.cc:648] HybridClock initialized: now 1786050638810954 us; error 46 us; skew 500 ppm
Aug 06 21:10:38 dist-test-slave-69rg krb5kdc[14588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050638, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:38.814085 14697 init.cc:377] Logged in from keytab as kudu/127.27.225.193@KRBTEST.COM (short username kudu)
I20260806 21:10:38.815053 14697 webserver.cc:460] Webserver started at http://127.27.225.193:45793/ using document root <none> and password file <none>
I20260806 21:10:38.815654 14697 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:38.815749 14697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:38.815987 14697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:38.818181 14697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "47d55f95f83f499084941fbeeb04710d"
format_stamp: "Formatted at 2026-08-06 21:10:38 on dist-test-slave-69rg"
server_key: "f485609337dd27321e13fbcfe1bc7238"
server_key_iv: "914831f206a39d7ee8e2eae31f183b96"
server_key_version: "encryptionkey@0"
I20260806 21:10:38.818776 14697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "47d55f95f83f499084941fbeeb04710d"
format_stamp: "Formatted at 2026-08-06 21:10:38 on dist-test-slave-69rg"
server_key: "f485609337dd27321e13fbcfe1bc7238"
server_key_iv: "914831f206a39d7ee8e2eae31f183b96"
server_key_version: "encryptionkey@0"
I20260806 21:10:38.822820 14697 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 21:10:38.825505 14712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:38.827036 14697 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:10:38.827240 14697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "47d55f95f83f499084941fbeeb04710d"
format_stamp: "Formatted at 2026-08-06 21:10:38 on dist-test-slave-69rg"
server_key: "f485609337dd27321e13fbcfe1bc7238"
server_key_iv: "914831f206a39d7ee8e2eae31f183b96"
server_key_version: "encryptionkey@0"
I20260806 21:10:38.827368 14697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:38.841974 14697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:38.845475 14697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:38.845729 14697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:38.846478 14697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:10:38.847376 14697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:10:38.847450 14697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:38.847522 14697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:10:38.847574 14697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:38.856770 14697 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:34415
I20260806 21:10:38.856797 14826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:34415 every 8 connection(s)
I20260806 21:10:38.857676 14697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:10:38.864279 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 14697
I20260806 21:10:38.864439 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:10:38.864778 28551 external_mini_cluster.cc:1428] Setting key deaf4ab91df70d183439d1e5cb965812
Aug 06 21:10:38 dist-test-slave-69rg krb5kdc[14588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050638, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:10:38.872947 14673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:38.859775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:49615 (local address 127.27.225.254:36421)
0806 21:10:38.859975 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:38.859980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:38.860766 (+ 786us) server_negotiation.cc:407] Connection header received
0806 21:10:38.862107 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:38.862110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:38.862201 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:38.862283 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:38.863500 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:38.864206 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:38.865656 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:38.866004 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:38.866126 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:38.868745 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:38.868751 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:38.868756 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:10:38.868790 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:38.870399 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:38.871168 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:38.871171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:38.871172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:38.871252 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:38.871700 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:38.871702 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:38.871703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:38.871893 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:38.871973 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:38.872704 (+ 731us) server_negotiation.cc:299] Negotiation successful
0806 21:10:38.872819 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 21:10:38.874433 14829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:38.860149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:36421 (local address 127.27.225.193:49615)
0806 21:10:38.860606 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:38.860652 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:38.861887 (+ 1235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:38.862474 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:38.862481 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:38.862850 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:10:38.863339 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:38.863353 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:38.864384 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:38.864416 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:38.865518 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:38.865531 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:38.866173 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:38.866177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:38.866369 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:38.867017 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:10:38.867034 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:10:38.868563 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:10:38.870532 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:10:38.870564 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:10:38.870583 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:10:38.871005 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:10:38.871362 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:10:38.871368 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:10:38.871373 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:10:38.871584 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:10:38.872013 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:10:38.872022 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:10:38.872398 (+ 376us) client_negotiation.cc:769] Sending connection context
0806 21:10:38.872653 (+ 255us) client_negotiation.cc:240] Negotiation successful
0806 21:10:38.873091 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":107,"threads_started":1}
I20260806 21:10:38.875974 14827 heartbeater.cc:347] Connected to a master server at 127.27.225.254:36421
WARNING: no policy specified for kudu/127.27.225.194@KRBTEST.COM; defaulting to no policy
I20260806 21:10:38.876281 14827 heartbeater.cc:464] Registering TS with master...
I20260806 21:10:38.876981 14827 heartbeater.cc:511] Master 127.27.225.254:36421 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.194@KRBTEST.COM" created.
I20260806 21:10:38.878917 14634 ts_manager.cc:194] Registered new tserver with Master: 47d55f95f83f499084941fbeeb04710d (127.27.225.193:34415)
I20260806 21:10:38.880584 14634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.193@KRBTEST.COM'} at 127.27.225.193:49615
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
I20260806 21:10:38.898329 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:36421
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:44529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:10:39.018566 14832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:39.018887 14832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:39.018934 14832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:39.023787 14832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:39.023870 14832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:39.023936 14832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:10:39.029647 14832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:44529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:36421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:39.030929 14832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:39.032155 14832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:39.042757 14837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:39.042757 14838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:39.042757 14840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:39.043020 14832 server_base.cc:1034] running on GCE node
I20260806 21:10:39.043594 14832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:39.044468 14832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:39.045768 14832 hybrid_clock.cc:648] HybridClock initialized: now 1786050639045720 us; error 64 us; skew 500 ppm
Aug 06 21:10:39 dist-test-slave-69rg krb5kdc[14588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050639, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:39.049010 14832 init.cc:377] Logged in from keytab as kudu/127.27.225.194@KRBTEST.COM (short username kudu)
I20260806 21:10:39.050168 14832 webserver.cc:460] Webserver started at http://127.27.225.194:38101/ using document root <none> and password file <none>
I20260806 21:10:39.050803 14832 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:39.050908 14832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:39.051158 14832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:39.053306 14832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "340715eeb4fa45d29fad5e50a0178301"
format_stamp: "Formatted at 2026-08-06 21:10:39 on dist-test-slave-69rg"
server_key: "fcc510b779552871da692f9ca19e7a2b"
server_key_iv: "68f8f5c21f06ea27d19d98456253632b"
server_key_version: "encryptionkey@0"
I20260806 21:10:39.053990 14832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "340715eeb4fa45d29fad5e50a0178301"
format_stamp: "Formatted at 2026-08-06 21:10:39 on dist-test-slave-69rg"
server_key: "fcc510b779552871da692f9ca19e7a2b"
server_key_iv: "68f8f5c21f06ea27d19d98456253632b"
server_key_version: "encryptionkey@0"
I20260806 21:10:39.057794 14832 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 21:10:39.060505 14847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:39.062127 14832 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:10:39.062319 14832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "340715eeb4fa45d29fad5e50a0178301"
format_stamp: "Formatted at 2026-08-06 21:10:39 on dist-test-slave-69rg"
server_key: "fcc510b779552871da692f9ca19e7a2b"
server_key_iv: "68f8f5c21f06ea27d19d98456253632b"
server_key_version: "encryptionkey@0"
I20260806 21:10:39.062454 14832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:39.087373 14832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:39.091172 14832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:39.091408 14832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:39.092056 14832 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:10:39.093060 14832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:10:39.093138 14832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:39.093230 14832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:10:39.093279 14832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:39.103978 14832 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:42869
I20260806 21:10:39.104029 14961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:42869 every 8 connection(s)
I20260806 21:10:39.104964 14832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:10:39.107033 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 14832
I20260806 21:10:39.107173 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:10:39.107486 28551 external_mini_cluster.cc:1428] Setting key d6ef3a9d537f025bf04305b68bb45001
Aug 06 21:10:39 dist-test-slave-69rg krb5kdc[14588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050639, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.225.195@KRBTEST.COM; defaulting to no policy
I20260806 21:10:39.120965 14673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:39.107009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:58543 (local address 127.27.225.254:36421)
0806 21:10:39.107158 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:39.107162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:39.108340 (+ 1178us) server_negotiation.cc:407] Connection header received
0806 21:10:39.109887 (+ 1547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:39.109891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:39.109947 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:39.110026 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:39.111198 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.111745 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.113537 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.113771 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.113927 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:39.116931 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:39.116934 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:39.116937 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:10:39.116962 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:39.118581 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:39.119219 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:39.119223 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:39.119225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:39.119314 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:39.119681 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:39.119685 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:39.119686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:39.119943 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:39.120048 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:39.120592 (+ 544us) server_negotiation.cc:299] Negotiation successful
0806 21:10:39.120788 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 21:10:39.121786 14964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:39.107565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:36421 (local address 127.27.225.194:58543)
0806 21:10:39.108145 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:39.108241 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:39.109640 (+ 1399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:39.110203 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:39.110210 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:39.110568 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:10:39.111045 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:39.111059 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.111887 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.111893 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:39.113385 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:39.113416 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.113971 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.113975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:39.114148 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:39.114834 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:10:39.114850 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:10:39.116744 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:10:39.118773 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:10:39.118779 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:10:39.118796 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:10:39.119088 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:10:39.119418 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:10:39.119421 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:10:39.119423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:10:39.119573 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:10:39.120060 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:10:39.120068 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:10:39.120342 (+ 274us) client_negotiation.cc:769] Sending connection context
0806 21:10:39.120534 (+ 192us) client_negotiation.cc:240] Negotiation successful
0806 21:10:39.120889 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":151,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.195@KRBTEST.COM" created.
I20260806 21:10:39.123126 14962 heartbeater.cc:347] Connected to a master server at 127.27.225.254:36421
I20260806 21:10:39.123498 14962 heartbeater.cc:464] Registering TS with master...
I20260806 21:10:39.124045 14962 heartbeater.cc:511] Master 127.27.225.254:36421 requested a full tablet report, sending...
I20260806 21:10:39.125223 14634 ts_manager.cc:194] Registered new tserver with Master: 340715eeb4fa45d29fad5e50a0178301 (127.27.225.194:42869)
I20260806 21:10:39.125886 14634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.194@KRBTEST.COM'} at 127.27.225.194:58543
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
I20260806 21:10:39.143939 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:36421
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:44529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:10:39.265578 14967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:39.266028 14967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:39.266131 14967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:39.270419 14967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:39.270524 14967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:39.270619 14967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:10:39.276216 14967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:44529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:36421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:39.277405 14967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:39.278522 14967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:39.289243 14973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:39.289281 14972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:39.289310 14975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:39.289402 14967 server_base.cc:1034] running on GCE node
I20260806 21:10:39.290068 14967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:39.290755 14967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:39.291956 14967 hybrid_clock.cc:648] HybridClock initialized: now 1786050639291936 us; error 33 us; skew 500 ppm
Aug 06 21:10:39 dist-test-slave-69rg krb5kdc[14588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050639, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:39.294894 14967 init.cc:377] Logged in from keytab as kudu/127.27.225.195@KRBTEST.COM (short username kudu)
I20260806 21:10:39.295748 14967 webserver.cc:460] Webserver started at http://127.27.225.195:37279/ using document root <none> and password file <none>
I20260806 21:10:39.296277 14967 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:39.296336 14967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:39.296512 14967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:39.298719 14967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "c491c7f7be464f1c8d491ade7b8fa5ff"
format_stamp: "Formatted at 2026-08-06 21:10:39 on dist-test-slave-69rg"
server_key: "08a9b333c9678c707f75eb2da45a101b"
server_key_iv: "808a59df851d15f1fb8c15b287b42c76"
server_key_version: "encryptionkey@0"
I20260806 21:10:39.299263 14967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "c491c7f7be464f1c8d491ade7b8fa5ff"
format_stamp: "Formatted at 2026-08-06 21:10:39 on dist-test-slave-69rg"
server_key: "08a9b333c9678c707f75eb2da45a101b"
server_key_iv: "808a59df851d15f1fb8c15b287b42c76"
server_key_version: "encryptionkey@0"
I20260806 21:10:39.303458 14967 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:10:39.306169 14982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:39.307754 14967 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:10:39.307909 14967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "c491c7f7be464f1c8d491ade7b8fa5ff"
format_stamp: "Formatted at 2026-08-06 21:10:39 on dist-test-slave-69rg"
server_key: "08a9b333c9678c707f75eb2da45a101b"
server_key_iv: "808a59df851d15f1fb8c15b287b42c76"
server_key_version: "encryptionkey@0"
I20260806 21:10:39.308043 14967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:39.347987 14967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:39.351578 14967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:39.351825 14967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:39.352501 14967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:10:39.353399 14967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:10:39.353456 14967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:39.353498 14967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:10:39.353556 14967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:39.363379 14967 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:40215
I20260806 21:10:39.363579 15098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:40215 every 8 connection(s)
I20260806 21:10:39.364315 14967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:10:39.373009 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 14967
I20260806 21:10:39.373194 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:10:39.373499 28551 external_mini_cluster.cc:1428] Setting key 22839919e34da65a555fc1078e703a31
Aug 06 21:10:39 dist-test-slave-69rg krb5kdc[14588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050639, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:10:39.379070 14673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:39.366357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:39651 (local address 127.27.225.254:36421)
0806 21:10:39.366524 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:39.366529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:39.367644 (+ 1115us) server_negotiation.cc:407] Connection header received
0806 21:10:39.369015 (+ 1371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:39.369018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:39.369070 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:39.369160 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:39.370405 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.370944 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.372140 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.372372 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.372458 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:39.375411 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:39.375415 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:39.375418 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:10:39.375448 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:39.376905 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:39.377482 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:39.377484 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:39.377486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:39.377559 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:39.377970 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:39.377973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:39.377974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:39.378203 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:39.378296 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:39.378844 (+ 548us) server_negotiation.cc:299] Negotiation successful
0806 21:10:39.378951 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 21:10:39.380007 15101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:39.366826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:36421 (local address 127.27.225.195:39651)
0806 21:10:39.367466 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:39.367521 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:39.368788 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:39.369301 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:39.369308 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:39.369677 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:10:39.370255 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:39.370270 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.371076 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.371079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:39.371989 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:39.371998 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.372518 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.372521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:39.372667 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:39.373363 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:10:39.373387 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:10:39.375252 (+ 1865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:10:39.377036 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:10:39.377041 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:10:39.377054 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:10:39.377385 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:10:39.377642 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:10:39.377645 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:10:39.377646 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:10:39.377792 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:10:39.378348 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:10:39.378354 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:10:39.378604 (+ 250us) client_negotiation.cc:769] Sending connection context
0806 21:10:39.378779 (+ 175us) client_negotiation.cc:240] Negotiation successful
0806 21:10:39.379051 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":111,"threads_started":1}
I20260806 21:10:39.381129 15099 heartbeater.cc:347] Connected to a master server at 127.27.225.254:36421
I20260806 21:10:39.381378 15099 heartbeater.cc:464] Registering TS with master...
I20260806 21:10:39.381978 15099 heartbeater.cc:511] Master 127.27.225.254:36421 requested a full tablet report, sending...
I20260806 21:10:39.383024 14634 ts_manager.cc:194] Registered new tserver with Master: c491c7f7be464f1c8d491ade7b8fa5ff (127.27.225.195:40215)
I20260806 21:10:39.383656 14634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.195@KRBTEST.COM'} at 127.27.225.195:39651
I20260806 21:10:39.387513 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:10:39.398103 14673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:39.389735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58898 (local address 127.27.225.254:36421)
0806 21:10:39.389894 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:39.389899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:39.390131 (+ 232us) server_negotiation.cc:407] Connection header received
0806 21:10:39.390287 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:39.390290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:39.390343 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:39.390429 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:39.391155 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.391694 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.392555 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.392796 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.392902 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:39.394654 (+ 1752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:39.394660 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:39.394663 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:10:39.394690 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:39.396091 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:39.396656 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:39.396660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:39.396662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:39.396751 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:39.397141 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:39.397148 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:39.397150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:39.397355 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:39.397526 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:39.397800 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 21:10:39.397917 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 21:10:39.401172 14634 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:10:39.404392 14634 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:10:39.419621 15110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:39.413249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:40215 (local address 127.0.0.1:45220)
0806 21:10:39.414069 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:39.414121 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:39.414266 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:39.415023 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:39.415030 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:39.415069 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:39.415266 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:39.415273 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.416466 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.416472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:39.417918 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:39.417930 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.419153 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.419157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:39.419242 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:39.419264 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 21:10:39.419345 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 21:10:39.419422 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":42,"spinlock_wait_cycles":7680,"thread_start_us":109,"threads_started":1}
I20260806 21:10:39.419660 15112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:39.413334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45220 (local address 127.27.225.195:40215)
0806 21:10:39.414208 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:39.414220 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:39.414244 (+ 24us) server_negotiation.cc:407] Connection header received
0806 21:10:39.414462 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:39.414468 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:39.414679 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:39.414839 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:39.415409 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.416338 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.418088 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.418953 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.419093 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:39.419219 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:39.419396 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 21:10:39.419505 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":764,"thread_start_us":108,"threads_started":1}
I20260806 21:10:39.420126 15113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:39.413614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50224 (local address 127.27.225.194:42869)
0806 21:10:39.414001 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:39.414013 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:39.414473 (+ 460us) server_negotiation.cc:407] Connection header received
0806 21:10:39.414576 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:39.414585 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:39.414796 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:39.414959 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:39.415393 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.416596 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.418101 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.418808 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.418939 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:39.419070 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:39.419511 (+ 441us) server_negotiation.cc:299] Negotiation successful
0806 21:10:39.419617 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":101,"threads_started":1}
I20260806 21:10:39.420222 15114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:39.414046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:34415 (local address 127.0.0.1:57138)
0806 21:10:39.414868 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:39.414881 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:39.414954 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:39.415574 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:39.415576 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:39.415596 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:39.415702 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:39.415709 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.417168 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.417171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:39.418487 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:39.418499 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.419799 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.419805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:39.419911 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:39.419932 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 21:10:39.420011 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 21:10:39.420091 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"thread_start_us":87,"threads_started":1}
I20260806 21:10:39.420835 15115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:39.414478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57138 (local address 127.27.225.193:34415)
0806 21:10:39.414904 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:39.414912 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:39.414935 (+ 23us) server_negotiation.cc:407] Connection header received
0806 21:10:39.415117 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:39.415124 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:39.415303 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:39.415435 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:39.415845 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.417031 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.418647 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.419495 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.419602 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:39.419725 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:39.420079 (+ 354us) server_negotiation.cc:299] Negotiation successful
0806 21:10:39.420682 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":117,"threads_started":1}
I20260806 21:10:39.423396 14760 tablet_service.cc:1467] Processing CreateTablet for tablet 3c1b214238624e5f80fa2374b0adcf25 (DEFAULT_TABLE table=test-table [id=99a569708058411a903667caae774950]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:10:39.423264 15032 tablet_service.cc:1467] Processing CreateTablet for tablet 3c1b214238624e5f80fa2374b0adcf25 (DEFAULT_TABLE table=test-table [id=99a569708058411a903667caae774950]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:10:39.419993 15111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:39.413240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:42869 (local address 127.0.0.1:50224)
0806 21:10:39.414227 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:39.414245 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:39.414359 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:39.415026 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:39.415030 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:39.415070 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:39.415265 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:39.415271 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.416755 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.416758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:39.417936 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:39.417942 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.418949 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.418953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:39.419083 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:39.419132 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 21:10:39.419252 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 21:10:39.419424 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":58,"spinlock_wait_cycles":3840,"thread_start_us":102,"threads_started":1}
I20260806 21:10:39.424422 14760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c1b214238624e5f80fa2374b0adcf25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:39.424422 15032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c1b214238624e5f80fa2374b0adcf25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:39.426676 14896 tablet_service.cc:1467] Processing CreateTablet for tablet 3c1b214238624e5f80fa2374b0adcf25 (DEFAULT_TABLE table=test-table [id=99a569708058411a903667caae774950]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:10:39.427524 14896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c1b214238624e5f80fa2374b0adcf25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:39.431761 15116 tablet_bootstrap.cc:492] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d: Bootstrap starting.
I20260806 21:10:39.432503 15117 tablet_bootstrap.cc:492] T 3c1b214238624e5f80fa2374b0adcf25 P c491c7f7be464f1c8d491ade7b8fa5ff: Bootstrap starting.
I20260806 21:10:39.434454 15117 tablet_bootstrap.cc:654] T 3c1b214238624e5f80fa2374b0adcf25 P c491c7f7be464f1c8d491ade7b8fa5ff: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:39.434590 15116 tablet_bootstrap.cc:654] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:39.434746 15118 tablet_bootstrap.cc:492] T 3c1b214238624e5f80fa2374b0adcf25 P 340715eeb4fa45d29fad5e50a0178301: Bootstrap starting.
I20260806 21:10:39.435933 15117 log.cc:824] T 3c1b214238624e5f80fa2374b0adcf25 P c491c7f7be464f1c8d491ade7b8fa5ff: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:39.435938 15116 log.cc:824] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:39.437153 15118 tablet_bootstrap.cc:654] T 3c1b214238624e5f80fa2374b0adcf25 P 340715eeb4fa45d29fad5e50a0178301: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:39.438272 15118 log.cc:824] T 3c1b214238624e5f80fa2374b0adcf25 P 340715eeb4fa45d29fad5e50a0178301: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:39.438288 15116 tablet_bootstrap.cc:492] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d: No bootstrap required, opened a new log
I20260806 21:10:39.438288 15117 tablet_bootstrap.cc:492] T 3c1b214238624e5f80fa2374b0adcf25 P c491c7f7be464f1c8d491ade7b8fa5ff: No bootstrap required, opened a new log
I20260806 21:10:39.438514 15117 ts_tablet_manager.cc:1397] T 3c1b214238624e5f80fa2374b0adcf25 P c491c7f7be464f1c8d491ade7b8fa5ff: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 21:10:39.438514 15116 ts_tablet_manager.cc:1397] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 21:10:39.440132 15118 tablet_bootstrap.cc:492] T 3c1b214238624e5f80fa2374b0adcf25 P 340715eeb4fa45d29fad5e50a0178301: No bootstrap required, opened a new log
I20260806 21:10:39.440337 15118 ts_tablet_manager.cc:1397] T 3c1b214238624e5f80fa2374b0adcf25 P 340715eeb4fa45d29fad5e50a0178301: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:10:39.443078 15117 raft_consensus.cc:348] T 3c1b214238624e5f80fa2374b0adcf25 P c491c7f7be464f1c8d491ade7b8fa5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c491c7f7be464f1c8d491ade7b8fa5ff" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40215 } } peers { permanent_uuid: "340715eeb4fa45d29fad5e50a0178301" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42869 } } peers { permanent_uuid: "47d55f95f83f499084941fbeeb04710d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 34415 } }
I20260806 21:10:39.443404 15117 raft_consensus.cc:374] T 3c1b214238624e5f80fa2374b0adcf25 P c491c7f7be464f1c8d491ade7b8fa5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:39.443467 15117 raft_consensus.cc:729] T 3c1b214238624e5f80fa2374b0adcf25 P c491c7f7be464f1c8d491ade7b8fa5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c491c7f7be464f1c8d491ade7b8fa5ff, State: Initialized, Role: FOLLOWER
I20260806 21:10:39.443339 15116 raft_consensus.cc:348] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c491c7f7be464f1c8d491ade7b8fa5ff" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40215 } } peers { permanent_uuid: "340715eeb4fa45d29fad5e50a0178301" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42869 } } peers { permanent_uuid: "47d55f95f83f499084941fbeeb04710d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 34415 } }
I20260806 21:10:39.443635 15116 raft_consensus.cc:374] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:39.443706 15116 raft_consensus.cc:729] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47d55f95f83f499084941fbeeb04710d, State: Initialized, Role: FOLLOWER
I20260806 21:10:39.443569 15118 raft_consensus.cc:348] T 3c1b214238624e5f80fa2374b0adcf25 P 340715eeb4fa45d29fad5e50a0178301 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c491c7f7be464f1c8d491ade7b8fa5ff" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40215 } } peers { permanent_uuid: "340715eeb4fa45d29fad5e50a0178301" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42869 } } peers { permanent_uuid: "47d55f95f83f499084941fbeeb04710d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 34415 } }
I20260806 21:10:39.443849 15118 raft_consensus.cc:374] T 3c1b214238624e5f80fa2374b0adcf25 P 340715eeb4fa45d29fad5e50a0178301 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:39.443967 15118 raft_consensus.cc:729] T 3c1b214238624e5f80fa2374b0adcf25 P 340715eeb4fa45d29fad5e50a0178301 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 340715eeb4fa45d29fad5e50a0178301, State: Initialized, Role: FOLLOWER
I20260806 21:10:39.444015 15117 consensus_queue.cc:260] T 3c1b214238624e5f80fa2374b0adcf25 P c491c7f7be464f1c8d491ade7b8fa5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c491c7f7be464f1c8d491ade7b8fa5ff" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40215 } } peers { permanent_uuid: "340715eeb4fa45d29fad5e50a0178301" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42869 } } peers { permanent_uuid: "47d55f95f83f499084941fbeeb04710d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 34415 } }
I20260806 21:10:39.444099 15116 consensus_queue.cc:260] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c491c7f7be464f1c8d491ade7b8fa5ff" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40215 } } peers { permanent_uuid: "340715eeb4fa45d29fad5e50a0178301" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42869 } } peers { permanent_uuid: "47d55f95f83f499084941fbeeb04710d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 34415 } }
I20260806 21:10:39.444470 15118 consensus_queue.cc:260] T 3c1b214238624e5f80fa2374b0adcf25 P 340715eeb4fa45d29fad5e50a0178301 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c491c7f7be464f1c8d491ade7b8fa5ff" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40215 } } peers { permanent_uuid: "340715eeb4fa45d29fad5e50a0178301" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42869 } } peers { permanent_uuid: "47d55f95f83f499084941fbeeb04710d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 34415 } }
I20260806 21:10:39.445111 15099 heartbeater.cc:503] Master 127.27.225.254:36421 was elected leader, sending a full tablet report...
I20260806 21:10:39.445214 15117 ts_tablet_manager.cc:1428] T 3c1b214238624e5f80fa2374b0adcf25 P c491c7f7be464f1c8d491ade7b8fa5ff: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 21:10:39.445212 15116 ts_tablet_manager.cc:1428] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 21:10:39.445431 15118 ts_tablet_manager.cc:1428] T 3c1b214238624e5f80fa2374b0adcf25 P 340715eeb4fa45d29fad5e50a0178301: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 21:10:39.445719 14962 heartbeater.cc:503] Master 127.27.225.254:36421 was elected leader, sending a full tablet report...
I20260806 21:10:39.445725 14827 heartbeater.cc:503] Master 127.27.225.254:36421 was elected leader, sending a full tablet report...
W20260806 21:10:39.606271 14963 tablet.cc:2369] T 3c1b214238624e5f80fa2374b0adcf25 P 340715eeb4fa45d29fad5e50a0178301: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 21:10:39.609145 14828 tablet.cc:2369] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 21:10:39.615319 15100 tablet.cc:2369] T 3c1b214238624e5f80fa2374b0adcf25 P c491c7f7be464f1c8d491ade7b8fa5ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:10:39.701073 15122 raft_consensus.cc:482] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:10:39.701347 15122 raft_consensus.cc:504] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c491c7f7be464f1c8d491ade7b8fa5ff" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40215 } } peers { permanent_uuid: "340715eeb4fa45d29fad5e50a0178301" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42869 } } peers { permanent_uuid: "47d55f95f83f499084941fbeeb04710d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 34415 } }
I20260806 21:10:39.702596 15122 leader_election.cc:288] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c491c7f7be464f1c8d491ade7b8fa5ff (127.27.225.195:40215), 340715eeb4fa45d29fad5e50a0178301 (127.27.225.194:42869)
I20260806 21:10:39.707716 15125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:39.702943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:42869 (local address 127.27.225.193:52941)
0806 21:10:39.703471 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:39.703489 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:39.703608 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:39.703936 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:39.703938 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:39.703954 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:39.704099 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:39.704108 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.704930 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.704936 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:39.706564 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:39.706575 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.707243 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.707245 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:39.707341 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:39.707361 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 21:10:39.707462 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 21:10:39.707534 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":155,"thread_start_us":134,"threads_started":1}
I20260806 21:10:39.707695 15113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:39.703011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:52941 (local address 127.27.225.194:42869)
0806 21:10:39.703171 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:39.703176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:39.703607 (+ 431us) server_negotiation.cc:407] Connection header received
0806 21:10:39.703747 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:39.703750 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:39.703827 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:39.703914 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:39.704239 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.704815 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.706716 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.707120 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.707235 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:39.707275 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:39.707486 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 21:10:39.707568 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 21:10:39.708454 15126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:39.702944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:40215 (local address 127.27.225.193:52753)
0806 21:10:39.703342 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:39.703355 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:39.703463 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:39.703761 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:39.703764 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:39.703780 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:39.703942 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:39.703949 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.704975 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.704978 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:39.706564 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:39.706575 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.708032 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.708036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:39.708136 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:39.708157 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 21:10:39.708239 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 21:10:39.708304 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":141,"thread_start_us":123,"threads_started":1}
I20260806 21:10:39.708468 15112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:39.703022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:52753 (local address 127.27.225.195:40215)
0806 21:10:39.703167 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:39.703172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:39.703460 (+ 288us) server_negotiation.cc:407] Connection header received
0806 21:10:39.703570 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:39.703575 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:39.703637 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:39.703733 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:39.704078 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.704842 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.707332 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.707739 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.707905 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:39.707964 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:39.708283 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 21:10:39.708360 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 21:10:39.708559 14916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c1b214238624e5f80fa2374b0adcf25" candidate_uuid: "47d55f95f83f499084941fbeeb04710d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "340715eeb4fa45d29fad5e50a0178301" is_pre_election: true
I20260806 21:10:39.708904 14916 raft_consensus.cc:2454] T 3c1b214238624e5f80fa2374b0adcf25 P 340715eeb4fa45d29fad5e50a0178301 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47d55f95f83f499084941fbeeb04710d in term 0.
I20260806 21:10:39.709081 15052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c1b214238624e5f80fa2374b0adcf25" candidate_uuid: "47d55f95f83f499084941fbeeb04710d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c491c7f7be464f1c8d491ade7b8fa5ff" is_pre_election: true
I20260806 21:10:39.709384 15052 raft_consensus.cc:2454] T 3c1b214238624e5f80fa2374b0adcf25 P c491c7f7be464f1c8d491ade7b8fa5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47d55f95f83f499084941fbeeb04710d in term 0.
I20260806 21:10:39.709488 14714 leader_election.cc:302] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 340715eeb4fa45d29fad5e50a0178301, 47d55f95f83f499084941fbeeb04710d; no voters:
I20260806 21:10:39.709798 15122 raft_consensus.cc:2781] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:10:39.709956 15122 raft_consensus.cc:482] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:10:39.710019 15122 raft_consensus.cc:3037] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:39.711076 15122 raft_consensus.cc:504] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c491c7f7be464f1c8d491ade7b8fa5ff" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40215 } } peers { permanent_uuid: "340715eeb4fa45d29fad5e50a0178301" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42869 } } peers { permanent_uuid: "47d55f95f83f499084941fbeeb04710d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 34415 } }
I20260806 21:10:39.711459 15122 leader_election.cc:288] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d [CANDIDATE]: Term 1 election: Requested vote from peers c491c7f7be464f1c8d491ade7b8fa5ff (127.27.225.195:40215), 340715eeb4fa45d29fad5e50a0178301 (127.27.225.194:42869)
I20260806 21:10:39.711884 15052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c1b214238624e5f80fa2374b0adcf25" candidate_uuid: "47d55f95f83f499084941fbeeb04710d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c491c7f7be464f1c8d491ade7b8fa5ff"
I20260806 21:10:39.712065 15052 raft_consensus.cc:3037] T 3c1b214238624e5f80fa2374b0adcf25 P c491c7f7be464f1c8d491ade7b8fa5ff [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:39.713287 15052 raft_consensus.cc:2454] T 3c1b214238624e5f80fa2374b0adcf25 P c491c7f7be464f1c8d491ade7b8fa5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47d55f95f83f499084941fbeeb04710d in term 1.
I20260806 21:10:39.713666 14715 leader_election.cc:302] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47d55f95f83f499084941fbeeb04710d, c491c7f7be464f1c8d491ade7b8fa5ff; no voters:
I20260806 21:10:39.713675 14916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c1b214238624e5f80fa2374b0adcf25" candidate_uuid: "47d55f95f83f499084941fbeeb04710d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "340715eeb4fa45d29fad5e50a0178301"
I20260806 21:10:39.713809 14916 raft_consensus.cc:3037] T 3c1b214238624e5f80fa2374b0adcf25 P 340715eeb4fa45d29fad5e50a0178301 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:39.713901 15122 raft_consensus.cc:2781] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:10:39.714169 15122 raft_consensus.cc:686] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d [term 1 LEADER]: Becoming Leader. State: Replica: 47d55f95f83f499084941fbeeb04710d, State: Running, Role: LEADER
I20260806 21:10:39.714507 15122 consensus_queue.cc:237] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c491c7f7be464f1c8d491ade7b8fa5ff" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40215 } } peers { permanent_uuid: "340715eeb4fa45d29fad5e50a0178301" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42869 } } peers { permanent_uuid: "47d55f95f83f499084941fbeeb04710d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 34415 } }
I20260806 21:10:39.715133 14916 raft_consensus.cc:2454] T 3c1b214238624e5f80fa2374b0adcf25 P 340715eeb4fa45d29fad5e50a0178301 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47d55f95f83f499084941fbeeb04710d in term 1.
I20260806 21:10:39.717674 14632 catalog_manager.cc:5495] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d reported cstate change: term changed from 0 to 1, leader changed from <none> to 47d55f95f83f499084941fbeeb04710d (127.27.225.193). New cstate: current_term: 1 leader_uuid: "47d55f95f83f499084941fbeeb04710d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c491c7f7be464f1c8d491ade7b8fa5ff" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "340715eeb4fa45d29fad5e50a0178301" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47d55f95f83f499084941fbeeb04710d" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 34415 } health_report { overall_health: HEALTHY } } }
I20260806 21:10:39.785874 15115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:39.782462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57150 (local address 127.27.225.193:34415)
0806 21:10:39.782607 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:39.782616 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:39.782629 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:10:39.782704 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:39.782706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:39.782760 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:39.782869 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:10:39.783099 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.783592 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.784412 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:39.784620 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:39.784699 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:39.785024 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:10:39.785435 (+ 411us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:10:39.785548 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:39.785661 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 21:10:39.785697 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 21:10:39.792501 15052 raft_consensus.cc:1261] T 3c1b214238624e5f80fa2374b0adcf25 P c491c7f7be464f1c8d491ade7b8fa5ff [term 1 FOLLOWER]: Refusing update from remote peer 47d55f95f83f499084941fbeeb04710d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:10:39.792500 14916 raft_consensus.cc:1261] T 3c1b214238624e5f80fa2374b0adcf25 P 340715eeb4fa45d29fad5e50a0178301 [term 1 FOLLOWER]: Refusing update from remote peer 47d55f95f83f499084941fbeeb04710d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:10:39.793303 15127 consensus_queue.cc:1035] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d [LEADER]: Connected to new peer: Peer: permanent_uuid: "340715eeb4fa45d29fad5e50a0178301" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 42869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:10:39.793515 15122 consensus_queue.cc:1035] T 3c1b214238624e5f80fa2374b0adcf25 P 47d55f95f83f499084941fbeeb04710d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c491c7f7be464f1c8d491ade7b8fa5ff" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:10:39.805431 15131 mvcc.cc:205] Tried to move back new op lower bound from 7315663420582223872 to 7315663420273119232. Current Snapshot: MvccSnapshot[applied={T|T < 7315663420582223872}]
I20260806 21:10:39.812480 14632 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:10:39.812671 14632 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:10:39.815523 14632 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c36b72c72094cfa8f01546b15063e6f: Sending DeleteTablet for 3 replicas of tablet 3c1b214238624e5f80fa2374b0adcf25
I20260806 21:10:39.816512 15032 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c1b214238624e5f80fa2374b0adcf25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:10:39 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:45220
I20260806 21:10:39.816556 14896 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c1b214238624e5f80fa2374b0adcf25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:10:39 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:50224
I20260806 21:10:39.817198 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 14697
I20260806 21:10:39.817233 14760 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c1b214238624e5f80fa2374b0adcf25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:10:39 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:57138
I20260806 21:10:39.818835 15140 tablet_replica.cc:330] T 3c1b214238624e5f80fa2374b0adcf25 P c491c7f7be464f1c8d491ade7b8fa5ff: stopping tablet replica
I20260806 21:10:39.819223 15140 raft_consensus.cc:2229] T 3c1b214238624e5f80fa2374b0adcf25 P c491c7f7be464f1c8d491ade7b8fa5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:10:39.819573 15140 raft_consensus.cc:2258] T 3c1b214238624e5f80fa2374b0adcf25 P c491c7f7be464f1c8d491ade7b8fa5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:10:39.819988 15141 tablet_replica.cc:330] T 3c1b214238624e5f80fa2374b0adcf25 P 340715eeb4fa45d29fad5e50a0178301: stopping tablet replica
I20260806 21:10:39.820328 15141 raft_consensus.cc:2229] T 3c1b214238624e5f80fa2374b0adcf25 P 340715eeb4fa45d29fad5e50a0178301 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:10:39.820660 15141 raft_consensus.cc:2258] T 3c1b214238624e5f80fa2374b0adcf25 P 340715eeb4fa45d29fad5e50a0178301 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:10:39.821373 15140 ts_tablet_manager.cc:1905] T 3c1b214238624e5f80fa2374b0adcf25 P c491c7f7be464f1c8d491ade7b8fa5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:10:39.822193 15141 ts_tablet_manager.cc:1905] T 3c1b214238624e5f80fa2374b0adcf25 P 340715eeb4fa45d29fad5e50a0178301: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:10:39.824184 15140 ts_tablet_manager.cc:1918] T 3c1b214238624e5f80fa2374b0adcf25 P c491c7f7be464f1c8d491ade7b8fa5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:10:39.824316 15140 log.cc:1196] T 3c1b214238624e5f80fa2374b0adcf25 P c491c7f7be464f1c8d491ade7b8fa5ff: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/3c1b214238624e5f80fa2374b0adcf25
I20260806 21:10:39.824626 15140 ts_tablet_manager.cc:1939] T 3c1b214238624e5f80fa2374b0adcf25 P c491c7f7be464f1c8d491ade7b8fa5ff: Deleting consensus metadata
I20260806 21:10:39.825642 15141 ts_tablet_manager.cc:1918] T 3c1b214238624e5f80fa2374b0adcf25 P 340715eeb4fa45d29fad5e50a0178301: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:10:39.825750 15141 log.cc:1196] T 3c1b214238624e5f80fa2374b0adcf25 P 340715eeb4fa45d29fad5e50a0178301: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050378084738-28551-0/minicluster-data/ts-1/wal/wals/3c1b214238624e5f80fa2374b0adcf25
I20260806 21:10:39.825901 14620 catalog_manager.cc:4841] TS c491c7f7be464f1c8d491ade7b8fa5ff (127.27.225.195:40215): tablet 3c1b214238624e5f80fa2374b0adcf25 (table test-table [id=99a569708058411a903667caae774950]) successfully deleted
I20260806 21:10:39.826098 15141 ts_tablet_manager.cc:1939] T 3c1b214238624e5f80fa2374b0adcf25 P 340715eeb4fa45d29fad5e50a0178301: Deleting consensus metadata
W20260806 21:10:39.826843 14620 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.225.193:34415 (error 108)
I20260806 21:10:39.827597 14619 catalog_manager.cc:4841] TS 340715eeb4fa45d29fad5e50a0178301 (127.27.225.194:42869): tablet 3c1b214238624e5f80fa2374b0adcf25 (table test-table [id=99a569708058411a903667caae774950]) successfully deleted
I20260806 21:10:39.827914 15111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:39.827593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:34415 (local address 127.0.0.1:57158)
0806 21:10:39.827734 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:39.827810 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.193:34415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 21:10:39.828052 14620 catalog_manager.cc:4584] TS 47d55f95f83f499084941fbeeb04710d (127.27.225.193:34415): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3c1b214238624e5f80fa2374b0adcf25: Network error: Client connection negotiation failed: client connection to 127.27.225.193:34415: connect: Connection refused (error 111)
I20260806 21:10:39.828106 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 14832
I20260806 21:10:39.835873 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 14967
I20260806 21:10:39.842653 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 14602
2026-08-06T21:10:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2696 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T21:10:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:10:39Z Disabled control of system clock
I20260806 21:10:39.876606 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:41161
--webserver_interface=127.27.225.254
--webserver_port=0
--builtin_ntp_servers=127.27.225.212:44419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:41161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 21:10:40.003507 15146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:40.003837 15146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:40.003880 15146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:40.009068 15146 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:10:40.009197 15146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:40.009217 15146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:40.009238 15146 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:10:40.009256 15146 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:10:40.015486 15146 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:44419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:41161
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:41161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:40.016726 15146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:40.018194 15146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:40.027671 15153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:40.027803 15155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:40.027841 15152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:40.028025 15146 server_base.cc:1034] running on GCE node
I20260806 21:10:40.028780 15146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:40.030191 15146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:40.031424 15146 hybrid_clock.cc:648] HybridClock initialized: now 1786050640031397 us; error 50 us; skew 500 ppm
I20260806 21:10:40.033713 15146 webserver.cc:460] Webserver started at http://127.27.225.254:36309/ using document root <none> and password file <none>
I20260806 21:10:40.034499 15146 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:40.034608 15146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:40.034886 15146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:40.037297 15146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "d4742b2338c94451958c216ffcfdcb36"
format_stamp: "Formatted at 2026-08-06 21:10:40 on dist-test-slave-69rg"
server_key: "e84f98704dfa8e1b6126e6138a367803"
server_key_iv: "ce45342ac539ad06ce1c362a177643ba"
server_key_version: "encryptionkey@0"
I20260806 21:10:40.038038 15146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "d4742b2338c94451958c216ffcfdcb36"
format_stamp: "Formatted at 2026-08-06 21:10:40 on dist-test-slave-69rg"
server_key: "e84f98704dfa8e1b6126e6138a367803"
server_key_iv: "ce45342ac539ad06ce1c362a177643ba"
server_key_version: "encryptionkey@0"
I20260806 21:10:40.042737 15146 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 21:10:40.045552 15161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:40.046916 15146 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 21:10:40.047075 15146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "d4742b2338c94451958c216ffcfdcb36"
format_stamp: "Formatted at 2026-08-06 21:10:40 on dist-test-slave-69rg"
server_key: "e84f98704dfa8e1b6126e6138a367803"
server_key_iv: "ce45342ac539ad06ce1c362a177643ba"
server_key_version: "encryptionkey@0"
I20260806 21:10:40.047204 15146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:40.083346 15146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:40.084256 15146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:40.084483 15146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:40.092392 15146 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:41161
I20260806 21:10:40.092402 15215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:41161 every 8 connection(s)
I20260806 21:10:40.093398 15146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:10:40.094321 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 15146
I20260806 21:10:40.094468 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:10:40.094833 28551 external_mini_cluster.cc:1428] Setting key c265b25a67d0a4314b0ccc39a01c5229
I20260806 21:10:40.097342 15216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:40.104000 15216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4742b2338c94451958c216ffcfdcb36: Bootstrap starting.
I20260806 21:10:40.104687 15219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:40.096410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39402 (local address 127.27.225.254:41161)
0806 21:10:40.097086 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:40.097100 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:40.097137 (+ 37us) server_negotiation.cc:407] Connection header received
0806 21:10:40.097947 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:40.097966 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:40.098373 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:40.098648 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:40.099082 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.100064 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.101273 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.101724 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.101901 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:40.102826 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:40.102831 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:40.102842 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:40.103254 (+ 412us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:40.103313 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:40.103333 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:40.103416 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:40.103587 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 21:10:40.103893 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":162,"threads_started":1}
I20260806 21:10:40.106350 15216 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4742b2338c94451958c216ffcfdcb36: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:40.107087 15216 log.cc:824] T 00000000000000000000000000000000 P d4742b2338c94451958c216ffcfdcb36: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:40.108970 15216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4742b2338c94451958c216ffcfdcb36: No bootstrap required, opened a new log
I20260806 21:10:40.112381 15216 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4742b2338c94451958c216ffcfdcb36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4742b2338c94451958c216ffcfdcb36" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41161 } }
I20260806 21:10:40.112632 15216 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4742b2338c94451958c216ffcfdcb36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:40.112715 15216 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4742b2338c94451958c216ffcfdcb36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4742b2338c94451958c216ffcfdcb36, State: Initialized, Role: FOLLOWER
I20260806 21:10:40.113139 15216 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4742b2338c94451958c216ffcfdcb36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4742b2338c94451958c216ffcfdcb36" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41161 } }
I20260806 21:10:40.113296 15216 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4742b2338c94451958c216ffcfdcb36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:10:40.113390 15216 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4742b2338c94451958c216ffcfdcb36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:10:40.113512 15216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4742b2338c94451958c216ffcfdcb36 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:40.114530 15216 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4742b2338c94451958c216ffcfdcb36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4742b2338c94451958c216ffcfdcb36" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41161 } }
I20260806 21:10:40.114868 15216 leader_election.cc:302] T 00000000000000000000000000000000 P d4742b2338c94451958c216ffcfdcb36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4742b2338c94451958c216ffcfdcb36; no voters:
I20260806 21:10:40.115185 15216 leader_election.cc:288] T 00000000000000000000000000000000 P d4742b2338c94451958c216ffcfdcb36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:10:40.115315 15221 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4742b2338c94451958c216ffcfdcb36 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:10:40.115566 15221 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4742b2338c94451958c216ffcfdcb36 [term 1 LEADER]: Becoming Leader. State: Replica: d4742b2338c94451958c216ffcfdcb36, State: Running, Role: LEADER
I20260806 21:10:40.115890 15221 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4742b2338c94451958c216ffcfdcb36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4742b2338c94451958c216ffcfdcb36" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41161 } }
I20260806 21:10:40.116447 15216 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4742b2338c94451958c216ffcfdcb36 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:10:40.117511 15222 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4742b2338c94451958c216ffcfdcb36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4742b2338c94451958c216ffcfdcb36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4742b2338c94451958c216ffcfdcb36" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41161 } } }
I20260806 21:10:40.117648 15222 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4742b2338c94451958c216ffcfdcb36 [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:40.117913 15223 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4742b2338c94451958c216ffcfdcb36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4742b2338c94451958c216ffcfdcb36. Latest consensus state: current_term: 1 leader_uuid: "d4742b2338c94451958c216ffcfdcb36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4742b2338c94451958c216ffcfdcb36" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41161 } } }
I20260806 21:10:40.118026 15223 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4742b2338c94451958c216ffcfdcb36 [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:40.118081 15230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:10:40.121111 15230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:10:40.126806 15230 catalog_manager.cc:1347] Generated new cluster ID: fc2fcb254a604182b45648dbfc511573
I20260806 21:10:40.126881 15230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:10:40.152545 15230 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:10:40.153935 15230 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:10:40.159942 15230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4742b2338c94451958c216ffcfdcb36: Generated new TSK 0
I20260806 21:10:40.160812 15230 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:10:40.171991 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:41161
--builtin_ntp_servers=127.27.225.212:44419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 21:10:40.292101 15241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:40.292394 15241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:40.292434 15241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:40.296510 15241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:40.296581 15241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:40.296640 15241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:10:40.301759 15241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:44419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:41161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:40.302944 15241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:40.304682 15241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:40.315326 15246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:40.315339 15247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:40.315343 15249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:40.315815 15241 server_base.cc:1034] running on GCE node
I20260806 21:10:40.316246 15241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:40.317003 15241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:40.318166 15241 hybrid_clock.cc:648] HybridClock initialized: now 1786050640318148 us; error 30 us; skew 500 ppm
I20260806 21:10:40.320228 15241 webserver.cc:460] Webserver started at http://127.27.225.193:44329/ using document root <none> and password file <none>
I20260806 21:10:40.320818 15241 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:40.320883 15241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:40.321060 15241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:40.323314 15241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "32a3a545fa7e42318a50fe42110d50f4"
format_stamp: "Formatted at 2026-08-06 21:10:40 on dist-test-slave-69rg"
server_key: "15ba528c1559e36e78b33beb2f439b22"
server_key_iv: "8de973ad550ed6fdabbf0eaac0d53e76"
server_key_version: "encryptionkey@0"
I20260806 21:10:40.323977 15241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "32a3a545fa7e42318a50fe42110d50f4"
format_stamp: "Formatted at 2026-08-06 21:10:40 on dist-test-slave-69rg"
server_key: "15ba528c1559e36e78b33beb2f439b22"
server_key_iv: "8de973ad550ed6fdabbf0eaac0d53e76"
server_key_version: "encryptionkey@0"
I20260806 21:10:40.327940 15241 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:10:40.330698 15255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:40.331966 15241 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:10:40.332070 15241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "32a3a545fa7e42318a50fe42110d50f4"
format_stamp: "Formatted at 2026-08-06 21:10:40 on dist-test-slave-69rg"
server_key: "15ba528c1559e36e78b33beb2f439b22"
server_key_iv: "8de973ad550ed6fdabbf0eaac0d53e76"
server_key_version: "encryptionkey@0"
I20260806 21:10:40.332167 15241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:40.349035 15241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:40.349799 15241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:40.350025 15241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:40.350646 15241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:10:40.351658 15241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:10:40.351727 15241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:40.351799 15241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:10:40.351851 15241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:40.361110 15241 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:37061
I20260806 21:10:40.361132 15375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:37061 every 8 connection(s)
I20260806 21:10:40.362161 15241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:10:40.370129 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 15241
I20260806 21:10:40.370289 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:10:40.370594 28551 external_mini_cluster.cc:1428] Setting key 3f9078a63f73c944529911c10569b108
I20260806 21:10:40.372020 15219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:40.364088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:54301 (local address 127.27.225.254:41161)
0806 21:10:40.364232 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:40.364237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:40.365345 (+ 1108us) server_negotiation.cc:407] Connection header received
0806 21:10:40.366816 (+ 1471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:40.366819 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:40.366880 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:40.367003 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:40.367941 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.368487 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.369648 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.369913 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.370032 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:40.371426 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:40.371430 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:40.371433 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:40.371448 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:40.371472 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:40.371484 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:40.371572 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:40.371769 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 21:10:40.371838 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 21:10:40.372790 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:41161
--builtin_ntp_servers=127.27.225.212:44419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 21:10:40.373117 15378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:40.364487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41161 (local address 127.27.225.193:54301)
0806 21:10:40.365198 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:40.365240 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:40.366603 (+ 1363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:40.367196 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:40.367203 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:40.367280 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:10:40.367763 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:40.367775 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.368647 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.368650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:40.369521 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:40.369530 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.370078 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.370082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:40.370286 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:40.371032 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:10:40.371034 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:10:40.371075 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:10:40.371183 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:10:40.371604 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:10:40.371613 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:10:40.371618 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 21:10:40.371745 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 21:10:40.372078 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":146,"threads_started":1}
I20260806 21:10:40.374519 15376 heartbeater.cc:347] Connected to a master server at 127.27.225.254:41161
I20260806 21:10:40.374764 15376 heartbeater.cc:464] Registering TS with master...
I20260806 21:10:40.375358 15376 heartbeater.cc:511] Master 127.27.225.254:41161 requested a full tablet report, sending...
I20260806 21:10:40.376894 15179 ts_manager.cc:194] Registered new tserver with Master: 32a3a545fa7e42318a50fe42110d50f4 (127.27.225.193:37061)
I20260806 21:10:40.378203 15179 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.225.193:54301
W20260806 21:10:40.486095 15379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:40.486462 15379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:40.486541 15379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:40.490702 15379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:40.490792 15379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:40.490882 15379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:10:40.495720 15379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:44419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:41161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:40.496887 15379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:40.498363 15379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:40.508849 15385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:40.508853 15387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:40.508849 15384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:40.510133 15379 server_base.cc:1034] running on GCE node
I20260806 21:10:40.510653 15379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:40.511549 15379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:40.512760 15379 hybrid_clock.cc:648] HybridClock initialized: now 1786050640512734 us; error 42 us; skew 500 ppm
I20260806 21:10:40.514890 15379 webserver.cc:460] Webserver started at http://127.27.225.194:34351/ using document root <none> and password file <none>
I20260806 21:10:40.515520 15379 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:40.515643 15379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:40.515883 15379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:40.518527 15379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "34bb2ff52a1142598fd3a989d5b94d11"
format_stamp: "Formatted at 2026-08-06 21:10:40 on dist-test-slave-69rg"
server_key: "86a7e9ec683c104ecf742b56e119e0b2"
server_key_iv: "011fee83ef37092b8ce78be8003a79ff"
server_key_version: "encryptionkey@0"
I20260806 21:10:40.519170 15379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "34bb2ff52a1142598fd3a989d5b94d11"
format_stamp: "Formatted at 2026-08-06 21:10:40 on dist-test-slave-69rg"
server_key: "86a7e9ec683c104ecf742b56e119e0b2"
server_key_iv: "011fee83ef37092b8ce78be8003a79ff"
server_key_version: "encryptionkey@0"
I20260806 21:10:40.523247 15379 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:10:40.526075 15393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:40.527410 15379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 21:10:40.527575 15379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "34bb2ff52a1142598fd3a989d5b94d11"
format_stamp: "Formatted at 2026-08-06 21:10:40 on dist-test-slave-69rg"
server_key: "86a7e9ec683c104ecf742b56e119e0b2"
server_key_iv: "011fee83ef37092b8ce78be8003a79ff"
server_key_version: "encryptionkey@0"
I20260806 21:10:40.527731 15379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:40.543483 15379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:40.544252 15379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:40.544472 15379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:40.545118 15379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:10:40.546238 15379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:10:40.546314 15379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:40.546386 15379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:10:40.546446 15379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:40.556138 15379 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:43019
I20260806 21:10:40.556164 15510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:43019 every 8 connection(s)
I20260806 21:10:40.557108 15379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:10:40.560633 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 15379
I20260806 21:10:40.560739 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:10:40.561055 28551 external_mini_cluster.cc:1428] Setting key ac8dc3c642163a64e55e017ccb33ca98
I20260806 21:10:40.563333 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:41161
--builtin_ntp_servers=127.27.225.212:44419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 21:10:40.567169 15219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:40.559091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:37915 (local address 127.27.225.254:41161)
0806 21:10:40.559258 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:40.559263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:40.560393 (+ 1130us) server_negotiation.cc:407] Connection header received
0806 21:10:40.562080 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:40.562084 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:40.562143 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:40.562231 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:40.563267 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.563908 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.565069 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.565302 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.565418 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:40.566625 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:40.566629 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:40.566631 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:40.566652 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:40.566671 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:40.566679 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:40.566758 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:40.566928 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 21:10:40.567021 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 21:10:40.568039 15513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:40.559561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41161 (local address 127.27.225.194:37915)
0806 21:10:40.560214 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:40.560254 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:40.561880 (+ 1626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:40.562439 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:40.562446 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:40.562558 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:10:40.563123 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:40.563136 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.564044 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.564048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:40.564937 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:40.564946 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.565459 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.565463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:40.565633 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:40.566340 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:10:40.566341 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:10:40.566382 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:10:40.566448 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:10:40.566773 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:10:40.566793 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:10:40.566798 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 21:10:40.566927 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 21:10:40.567222 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":117,"threads_started":1}
I20260806 21:10:40.569367 15511 heartbeater.cc:347] Connected to a master server at 127.27.225.254:41161
I20260806 21:10:40.569612 15511 heartbeater.cc:464] Registering TS with master...
I20260806 21:10:40.570206 15511 heartbeater.cc:511] Master 127.27.225.254:41161 requested a full tablet report, sending...
I20260806 21:10:40.571322 15179 ts_manager.cc:194] Registered new tserver with Master: 34bb2ff52a1142598fd3a989d5b94d11 (127.27.225.194:43019)
I20260806 21:10:40.571909 15179 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.225.194:37915
W20260806 21:10:40.683131 15514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:40.683467 15514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:40.683540 15514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:40.688024 15514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:40.688115 15514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:40.688208 15514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:10:40.693218 15514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:44419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:41161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:40.694454 15514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:40.695842 15514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:40.705490 15520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:40.705547 15522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:40.705526 15519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:40.705683 15514 server_base.cc:1034] running on GCE node
I20260806 21:10:40.706226 15514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:40.706884 15514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:40.708078 15514 hybrid_clock.cc:648] HybridClock initialized: now 1786050640708071 us; error 32 us; skew 500 ppm
I20260806 21:10:40.710093 15514 webserver.cc:460] Webserver started at http://127.27.225.195:42533/ using document root <none> and password file <none>
I20260806 21:10:40.710696 15514 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:40.710790 15514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:40.711024 15514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:40.713091 15514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "ce781f06852c4fbca814789d0580be0b"
format_stamp: "Formatted at 2026-08-06 21:10:40 on dist-test-slave-69rg"
server_key: "a25752d58afc7b0b5886899544d52251"
server_key_iv: "104cb877be8aa1f28f00ee9f7a1625af"
server_key_version: "encryptionkey@0"
I20260806 21:10:40.713745 15514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "ce781f06852c4fbca814789d0580be0b"
format_stamp: "Formatted at 2026-08-06 21:10:40 on dist-test-slave-69rg"
server_key: "a25752d58afc7b0b5886899544d52251"
server_key_iv: "104cb877be8aa1f28f00ee9f7a1625af"
server_key_version: "encryptionkey@0"
I20260806 21:10:40.717935 15514 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:10:40.720701 15528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:40.721993 15514 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:10:40.722127 15514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "ce781f06852c4fbca814789d0580be0b"
format_stamp: "Formatted at 2026-08-06 21:10:40 on dist-test-slave-69rg"
server_key: "a25752d58afc7b0b5886899544d52251"
server_key_iv: "104cb877be8aa1f28f00ee9f7a1625af"
server_key_version: "encryptionkey@0"
I20260806 21:10:40.722244 15514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:40.742612 15514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:40.743386 15514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:40.743604 15514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:40.744244 15514 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:10:40.745270 15514 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:10:40.745352 15514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:40.745426 15514 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:10:40.745471 15514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:40.754822 15514 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:45925
I20260806 21:10:40.754876 15645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:45925 every 8 connection(s)
I20260806 21:10:40.755810 15514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:10:40.761472 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 15514
I20260806 21:10:40.761564 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:10:40.762004 28551 external_mini_cluster.cc:1428] Setting key 887d78ffa0d6512172aca3bf6eff087b
I20260806 21:10:40.766642 15219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:40.757926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:58249 (local address 127.27.225.254:41161)
0806 21:10:40.758115 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:40.758120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:40.759328 (+ 1208us) server_negotiation.cc:407] Connection header received
0806 21:10:40.760851 (+ 1523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:40.760855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:40.760923 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:40.761017 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:40.762182 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.762883 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.764334 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.764616 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.764710 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:40.766096 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:40.766099 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:40.766101 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:40.766116 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:40.766137 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:40.766146 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:40.766253 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:40.766438 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 21:10:40.766517 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 21:10:40.767707 15648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:40.758360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:41161 (local address 127.27.225.195:58249)
0806 21:10:40.759110 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:40.759158 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:40.760588 (+ 1430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:40.761231 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:40.761239 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:40.761336 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:10:40.762006 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:40.762025 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.763031 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.763035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:40.764168 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:40.764206 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.764807 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.764811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:40.764982 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:40.765684 (+ 702us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:10:40.765686 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:10:40.765735 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:10:40.765866 (+ 131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:10:40.766263 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:10:40.766273 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:10:40.766279 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 21:10:40.766407 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 21:10:40.766758 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":141,"threads_started":1}
I20260806 21:10:40.769160 15646 heartbeater.cc:347] Connected to a master server at 127.27.225.254:41161
I20260806 21:10:40.769469 15646 heartbeater.cc:464] Registering TS with master...
I20260806 21:10:40.770130 15646 heartbeater.cc:511] Master 127.27.225.254:41161 requested a full tablet report, sending...
I20260806 21:10:40.771411 15179 ts_manager.cc:194] Registered new tserver with Master: ce781f06852c4fbca814789d0580be0b (127.27.225.195:45925)
I20260806 21:10:40.772060 15179 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.225.195:58249
I20260806 21:10:40.776522 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:10:40.782946 15219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:40.778853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39418 (local address 127.27.225.254:41161)
0806 21:10:40.779010 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:40.779014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:40.779100 (+ 86us) server_negotiation.cc:407] Connection header received
0806 21:10:40.779221 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:40.779223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:40.779264 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:40.779343 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:40.779665 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.780185 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.781032 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.781248 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.781339 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:40.782370 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:40.782373 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:40.782375 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:40.782389 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:40.782410 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:40.782419 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:40.782528 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:40.782671 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 21:10:40.782760 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 21:10:40.786235 15179 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39418:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:10:40.789451 15179 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:10:40.804715 15659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:40.798645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:43019 (local address 127.0.0.1:35298)
0806 21:10:40.799577 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:40.799641 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:40.799776 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:40.800450 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:40.800456 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:40.800479 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:40.800621 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:40.800631 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.801910 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.801914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:40.803284 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:40.803295 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.804133 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.804137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:40.804246 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:40.804284 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 21:10:40.804415 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 21:10:40.804545 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"mutex_wait_us":79,"thread_start_us":109,"threads_started":1}
I20260806 21:10:40.804719 15662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:40.799490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35298 (local address 127.27.225.194:43019)
0806 21:10:40.799959 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:40.799973 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:40.800001 (+ 28us) server_negotiation.cc:407] Connection header received
0806 21:10:40.800081 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:40.800088 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:40.800284 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:40.800446 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:40.800757 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.801753 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.803412 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.803995 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.804140 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:40.804341 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:40.804442 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 21:10:40.804545 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":138,"threads_started":1}
I20260806 21:10:40.805883 15658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:40.798696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:45925 (local address 127.0.0.1:51086)
0806 21:10:40.800310 (+ 1614us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:40.800324 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:40.800438 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:40.801149 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:40.801152 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:40.801169 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:40.801252 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:40.801258 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.802612 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.802615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:40.804074 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:40.804086 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.805408 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.805412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:40.805501 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:40.805524 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 21:10:40.805612 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 21:10:40.805709 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1524,"mutex_wait_us":214,"thread_start_us":110,"threads_started":1}
I20260806 21:10:40.805946 15660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:40.798915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51086 (local address 127.27.225.195:45925)
0806 21:10:40.800123 (+ 1208us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:40.800135 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:40.800540 (+ 405us) server_negotiation.cc:407] Connection header received
0806 21:10:40.800626 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:40.800632 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:40.800842 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:40.800989 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:40.801371 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.802498 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.804227 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.805282 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.805395 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:40.805524 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:40.805639 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 21:10:40.805735 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1094,"thread_start_us":148,"threads_started":1}
I20260806 21:10:40.806522 15661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:40.799472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59350 (local address 127.27.225.193:37061)
0806 21:10:40.799919 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:40.799933 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:40.800472 (+ 539us) server_negotiation.cc:407] Connection header received
0806 21:10:40.800697 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:40.800706 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:40.800861 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:40.801014 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:40.801241 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.802444 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.804226 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.805208 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.805307 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:40.805482 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:40.806262 (+ 780us) server_negotiation.cc:299] Negotiation successful
0806 21:10:40.806360 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":120,"threads_started":1}
I20260806 21:10:40.806784 15657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:40.798911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:37061 (local address 127.0.0.1:59350)
0806 21:10:40.800379 (+ 1468us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:40.800391 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:40.800539 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:40.800994 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:40.800997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:40.801015 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:40.801114 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:40.801124 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.802594 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.802599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:40.804074 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:40.804086 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.805588 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.805593 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:40.805723 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:40.805745 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 21:10:40.805835 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 21:10:40.805908 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1397,"mutex_wait_us":139,"thread_start_us":114,"threads_started":1}
I20260806 21:10:40.808025 15442 tablet_service.cc:1467] Processing CreateTablet for tablet 35c0a4590961409e8917357cd8a9f919 (DEFAULT_TABLE table=test-table [id=4333cdec07254958a84650f7ddf780df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:10:40.808223 15577 tablet_service.cc:1467] Processing CreateTablet for tablet 35c0a4590961409e8917357cd8a9f919 (DEFAULT_TABLE table=test-table [id=4333cdec07254958a84650f7ddf780df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:10:40.809125 15442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35c0a4590961409e8917357cd8a9f919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:40.809124 15577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35c0a4590961409e8917357cd8a9f919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:40.815678 15663 tablet_bootstrap.cc:492] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b: Bootstrap starting.
I20260806 21:10:40.817554 15664 tablet_bootstrap.cc:492] T 35c0a4590961409e8917357cd8a9f919 P 34bb2ff52a1142598fd3a989d5b94d11: Bootstrap starting.
I20260806 21:10:40.817181 15308 tablet_service.cc:1467] Processing CreateTablet for tablet 35c0a4590961409e8917357cd8a9f919 (DEFAULT_TABLE table=test-table [id=4333cdec07254958a84650f7ddf780df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:10:40.818040 15663 tablet_bootstrap.cc:654] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:40.818589 15308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35c0a4590961409e8917357cd8a9f919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:40.818986 15663 log.cc:824] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:40.819943 15664 tablet_bootstrap.cc:654] T 35c0a4590961409e8917357cd8a9f919 P 34bb2ff52a1142598fd3a989d5b94d11: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:40.820768 15664 log.cc:824] T 35c0a4590961409e8917357cd8a9f919 P 34bb2ff52a1142598fd3a989d5b94d11: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:40.821158 15663 tablet_bootstrap.cc:492] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b: No bootstrap required, opened a new log
I20260806 21:10:40.821372 15663 ts_tablet_manager.cc:1397] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260806 21:10:40.822976 15664 tablet_bootstrap.cc:492] T 35c0a4590961409e8917357cd8a9f919 P 34bb2ff52a1142598fd3a989d5b94d11: No bootstrap required, opened a new log
I20260806 21:10:40.823179 15664 ts_tablet_manager.cc:1397] T 35c0a4590961409e8917357cd8a9f919 P 34bb2ff52a1142598fd3a989d5b94d11: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:10:40.825366 15663 raft_consensus.cc:348] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34bb2ff52a1142598fd3a989d5b94d11" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43019 } } peers { permanent_uuid: "ce781f06852c4fbca814789d0580be0b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 45925 } } peers { permanent_uuid: "32a3a545fa7e42318a50fe42110d50f4" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37061 } }
I20260806 21:10:40.825616 15663 raft_consensus.cc:374] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:40.825697 15663 raft_consensus.cc:729] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce781f06852c4fbca814789d0580be0b, State: Initialized, Role: FOLLOWER
I20260806 21:10:40.826175 15663 consensus_queue.cc:260] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34bb2ff52a1142598fd3a989d5b94d11" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43019 } } peers { permanent_uuid: "ce781f06852c4fbca814789d0580be0b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 45925 } } peers { permanent_uuid: "32a3a545fa7e42318a50fe42110d50f4" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37061 } }
I20260806 21:10:40.826738 15667 tablet_bootstrap.cc:492] T 35c0a4590961409e8917357cd8a9f919 P 32a3a545fa7e42318a50fe42110d50f4: Bootstrap starting.
I20260806 21:10:40.827208 15663 ts_tablet_manager.cc:1428] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260806 21:10:40.827559 15646 heartbeater.cc:503] Master 127.27.225.254:41161 was elected leader, sending a full tablet report...
I20260806 21:10:40.827579 15664 raft_consensus.cc:348] T 35c0a4590961409e8917357cd8a9f919 P 34bb2ff52a1142598fd3a989d5b94d11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34bb2ff52a1142598fd3a989d5b94d11" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43019 } } peers { permanent_uuid: "ce781f06852c4fbca814789d0580be0b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 45925 } } peers { permanent_uuid: "32a3a545fa7e42318a50fe42110d50f4" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37061 } }
I20260806 21:10:40.827891 15664 raft_consensus.cc:374] T 35c0a4590961409e8917357cd8a9f919 P 34bb2ff52a1142598fd3a989d5b94d11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:40.827953 15664 raft_consensus.cc:729] T 35c0a4590961409e8917357cd8a9f919 P 34bb2ff52a1142598fd3a989d5b94d11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34bb2ff52a1142598fd3a989d5b94d11, State: Initialized, Role: FOLLOWER
I20260806 21:10:40.828388 15664 consensus_queue.cc:260] T 35c0a4590961409e8917357cd8a9f919 P 34bb2ff52a1142598fd3a989d5b94d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34bb2ff52a1142598fd3a989d5b94d11" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43019 } } peers { permanent_uuid: "ce781f06852c4fbca814789d0580be0b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 45925 } } peers { permanent_uuid: "32a3a545fa7e42318a50fe42110d50f4" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37061 } }
I20260806 21:10:40.829015 15511 heartbeater.cc:503] Master 127.27.225.254:41161 was elected leader, sending a full tablet report...
I20260806 21:10:40.829332 15667 tablet_bootstrap.cc:654] T 35c0a4590961409e8917357cd8a9f919 P 32a3a545fa7e42318a50fe42110d50f4: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:40.829383 15664 ts_tablet_manager.cc:1428] T 35c0a4590961409e8917357cd8a9f919 P 34bb2ff52a1142598fd3a989d5b94d11: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 21:10:40.830441 15667 log.cc:824] T 35c0a4590961409e8917357cd8a9f919 P 32a3a545fa7e42318a50fe42110d50f4: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:40.832752 15667 tablet_bootstrap.cc:492] T 35c0a4590961409e8917357cd8a9f919 P 32a3a545fa7e42318a50fe42110d50f4: No bootstrap required, opened a new log
I20260806 21:10:40.832988 15667 ts_tablet_manager.cc:1397] T 35c0a4590961409e8917357cd8a9f919 P 32a3a545fa7e42318a50fe42110d50f4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:10:40.836392 15667 raft_consensus.cc:348] T 35c0a4590961409e8917357cd8a9f919 P 32a3a545fa7e42318a50fe42110d50f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34bb2ff52a1142598fd3a989d5b94d11" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43019 } } peers { permanent_uuid: "ce781f06852c4fbca814789d0580be0b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 45925 } } peers { permanent_uuid: "32a3a545fa7e42318a50fe42110d50f4" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37061 } }
I20260806 21:10:40.836637 15667 raft_consensus.cc:374] T 35c0a4590961409e8917357cd8a9f919 P 32a3a545fa7e42318a50fe42110d50f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:40.836719 15667 raft_consensus.cc:729] T 35c0a4590961409e8917357cd8a9f919 P 32a3a545fa7e42318a50fe42110d50f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32a3a545fa7e42318a50fe42110d50f4, State: Initialized, Role: FOLLOWER
I20260806 21:10:40.837249 15667 consensus_queue.cc:260] T 35c0a4590961409e8917357cd8a9f919 P 32a3a545fa7e42318a50fe42110d50f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34bb2ff52a1142598fd3a989d5b94d11" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43019 } } peers { permanent_uuid: "ce781f06852c4fbca814789d0580be0b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 45925 } } peers { permanent_uuid: "32a3a545fa7e42318a50fe42110d50f4" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37061 } }
I20260806 21:10:40.837960 15376 heartbeater.cc:503] Master 127.27.225.254:41161 was elected leader, sending a full tablet report...
I20260806 21:10:40.838389 15667 ts_tablet_manager.cc:1428] T 35c0a4590961409e8917357cd8a9f919 P 32a3a545fa7e42318a50fe42110d50f4: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260806 21:10:40.863356 15377 tablet.cc:2369] T 35c0a4590961409e8917357cd8a9f919 P 32a3a545fa7e42318a50fe42110d50f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:10:40.914693 15668 raft_consensus.cc:482] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:10:40.914886 15668 raft_consensus.cc:504] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34bb2ff52a1142598fd3a989d5b94d11" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43019 } } peers { permanent_uuid: "ce781f06852c4fbca814789d0580be0b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 45925 } } peers { permanent_uuid: "32a3a545fa7e42318a50fe42110d50f4" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37061 } }
I20260806 21:10:40.916121 15668 leader_election.cc:288] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34bb2ff52a1142598fd3a989d5b94d11 (127.27.225.194:43019), 32a3a545fa7e42318a50fe42110d50f4 (127.27.225.193:37061)
I20260806 21:10:40.919951 15648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:40.916368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:37061 (local address 127.27.225.195:54577)
0806 21:10:40.916513 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:40.916530 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:40.916632 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:40.916911 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:40.916914 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:40.916934 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:40.917100 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:40.917107 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.918007 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.918011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:40.918853 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:40.918863 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.919509 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.919512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:40.919621 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:40.919643 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 21:10:40.919741 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 21:10:40.919812 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 21:10:40.919951 15661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:40.916484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:54577 (local address 127.27.225.193:37061)
0806 21:10:40.916609 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:40.916613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:40.916628 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:10:40.916738 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:40.916742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:40.916792 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:40.916892 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:40.917260 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.917902 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.918987 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.919390 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.919516 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:40.919567 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:40.919759 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 21:10:40.919828 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 21:10:40.920315 15662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:40.916455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:36939 (local address 127.27.225.194:43019)
0806 21:10:40.916596 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:40.916602 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:40.917054 (+ 452us) server_negotiation.cc:407] Connection header received
0806 21:10:40.917261 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:40.917264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:40.917319 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:40.917401 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:40.917742 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.918367 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.919360 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.919745 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.919875 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:40.919919 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:40.920113 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 21:10:40.920196 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 21:10:40.920645 15672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:40.916376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:43019 (local address 127.27.225.195:36939)
0806 21:10:40.916936 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:40.916948 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:40.917136 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:40.917421 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:40.917423 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:40.917439 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:40.917577 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:40.917582 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.918476 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.918481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:40.919233 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:40.919242 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.919867 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.919871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:40.919964 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:40.919985 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 21:10:40.920462 (+ 477us) client_negotiation.cc:240] Negotiation successful
0806 21:10:40.920536 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":168,"thread_start_us":174,"threads_started":1}
I20260806 21:10:40.920661 15329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35c0a4590961409e8917357cd8a9f919" candidate_uuid: "ce781f06852c4fbca814789d0580be0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32a3a545fa7e42318a50fe42110d50f4" is_pre_election: true
I20260806 21:10:40.920909 15329 raft_consensus.cc:2454] T 35c0a4590961409e8917357cd8a9f919 P 32a3a545fa7e42318a50fe42110d50f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce781f06852c4fbca814789d0580be0b in term 0.
I20260806 21:10:40.921320 15464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35c0a4590961409e8917357cd8a9f919" candidate_uuid: "ce781f06852c4fbca814789d0580be0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34bb2ff52a1142598fd3a989d5b94d11" is_pre_election: true
I20260806 21:10:40.921432 15532 leader_election.cc:302] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32a3a545fa7e42318a50fe42110d50f4, ce781f06852c4fbca814789d0580be0b; no voters:
I20260806 21:10:40.921610 15464 raft_consensus.cc:2454] T 35c0a4590961409e8917357cd8a9f919 P 34bb2ff52a1142598fd3a989d5b94d11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce781f06852c4fbca814789d0580be0b in term 0.
I20260806 21:10:40.921741 15668 raft_consensus.cc:2781] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:10:40.921885 15668 raft_consensus.cc:482] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:10:40.921954 15668 raft_consensus.cc:3037] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:40.923030 15668 raft_consensus.cc:504] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34bb2ff52a1142598fd3a989d5b94d11" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43019 } } peers { permanent_uuid: "ce781f06852c4fbca814789d0580be0b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 45925 } } peers { permanent_uuid: "32a3a545fa7e42318a50fe42110d50f4" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37061 } }
I20260806 21:10:40.923406 15668 leader_election.cc:288] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b [CANDIDATE]: Term 1 election: Requested vote from peers 34bb2ff52a1142598fd3a989d5b94d11 (127.27.225.194:43019), 32a3a545fa7e42318a50fe42110d50f4 (127.27.225.193:37061)
I20260806 21:10:40.923803 15464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35c0a4590961409e8917357cd8a9f919" candidate_uuid: "ce781f06852c4fbca814789d0580be0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34bb2ff52a1142598fd3a989d5b94d11"
I20260806 21:10:40.923822 15329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35c0a4590961409e8917357cd8a9f919" candidate_uuid: "ce781f06852c4fbca814789d0580be0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32a3a545fa7e42318a50fe42110d50f4"
I20260806 21:10:40.923951 15464 raft_consensus.cc:3037] T 35c0a4590961409e8917357cd8a9f919 P 34bb2ff52a1142598fd3a989d5b94d11 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:40.923951 15329 raft_consensus.cc:3037] T 35c0a4590961409e8917357cd8a9f919 P 32a3a545fa7e42318a50fe42110d50f4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:40.925292 15464 raft_consensus.cc:2454] T 35c0a4590961409e8917357cd8a9f919 P 34bb2ff52a1142598fd3a989d5b94d11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce781f06852c4fbca814789d0580be0b in term 1.
I20260806 21:10:40.925292 15329 raft_consensus.cc:2454] T 35c0a4590961409e8917357cd8a9f919 P 32a3a545fa7e42318a50fe42110d50f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce781f06852c4fbca814789d0580be0b in term 1.
I20260806 21:10:40.925642 15532 leader_election.cc:302] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32a3a545fa7e42318a50fe42110d50f4, ce781f06852c4fbca814789d0580be0b; no voters:
I20260806 21:10:40.925890 15668 raft_consensus.cc:2781] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:10:40.926182 15668 raft_consensus.cc:686] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b [term 1 LEADER]: Becoming Leader. State: Replica: ce781f06852c4fbca814789d0580be0b, State: Running, Role: LEADER
I20260806 21:10:40.926532 15668 consensus_queue.cc:237] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34bb2ff52a1142598fd3a989d5b94d11" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43019 } } peers { permanent_uuid: "ce781f06852c4fbca814789d0580be0b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 45925 } } peers { permanent_uuid: "32a3a545fa7e42318a50fe42110d50f4" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37061 } }
I20260806 21:10:40.930148 15179 catalog_manager.cc:5495] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b reported cstate change: term changed from 0 to 1, leader changed from <none> to ce781f06852c4fbca814789d0580be0b (127.27.225.195). New cstate: current_term: 1 leader_uuid: "ce781f06852c4fbca814789d0580be0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34bb2ff52a1142598fd3a989d5b94d11" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce781f06852c4fbca814789d0580be0b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 45925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32a3a545fa7e42318a50fe42110d50f4" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37061 } health_report { overall_health: UNKNOWN } } }
I20260806 21:10:40.969038 15660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:40.965325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51100 (local address 127.27.225.195:45925)
0806 21:10:40.965471 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:40.965475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:40.965490 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:10:40.965630 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:40.965633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:40.965690 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:40.965784 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:10:40.966090 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.966584 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.967450 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:40.967679 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:40.967763 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:40.968064 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:10:40.968504 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:10:40.968623 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:40.968818 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 21:10:40.968860 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 21:10:40.975080 15464 raft_consensus.cc:1261] T 35c0a4590961409e8917357cd8a9f919 P 34bb2ff52a1142598fd3a989d5b94d11 [term 1 FOLLOWER]: Refusing update from remote peer ce781f06852c4fbca814789d0580be0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:10:40.975099 15329 raft_consensus.cc:1261] T 35c0a4590961409e8917357cd8a9f919 P 32a3a545fa7e42318a50fe42110d50f4 [term 1 FOLLOWER]: Refusing update from remote peer ce781f06852c4fbca814789d0580be0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:10:40.975867 15668 consensus_queue.cc:1035] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "32a3a545fa7e42318a50fe42110d50f4" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 37061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:10:40.976078 15673 consensus_queue.cc:1035] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34bb2ff52a1142598fd3a989d5b94d11" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:10:40.983237 15676 mvcc.cc:205] Tried to move back new op lower bound from 7315663425426268160 to 7315663425237962752. Current Snapshot: MvccSnapshot[applied={T|T < 7315663425426268160}]
I20260806 21:10:40.984344 15678 mvcc.cc:205] Tried to move back new op lower bound from 7315663425426268160 to 7315663425237962752. Current Snapshot: MvccSnapshot[applied={T|T < 7315663425426268160}]
I20260806 21:10:40.987228 15677 mvcc.cc:205] Tried to move back new op lower bound from 7315663425426268160 to 7315663425237962752. Current Snapshot: MvccSnapshot[applied={T|T < 7315663425426268160 or (T in {7315663425426268160})}]
I20260806 21:10:40.993640 15177 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:10:40.993878 15177 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:10:40.996459 15177 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d4742b2338c94451958c216ffcfdcb36: Sending DeleteTablet for 3 replicas of tablet 35c0a4590961409e8917357cd8a9f919
I20260806 21:10:40.997427 15442 tablet_service.cc:1514] Processing DeleteTablet for tablet 35c0a4590961409e8917357cd8a9f919 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:10:40 UTC) from {username='slave'} at 127.0.0.1:35298
I20260806 21:10:40.997509 15577 tablet_service.cc:1514] Processing DeleteTablet for tablet 35c0a4590961409e8917357cd8a9f919 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:10:40 UTC) from {username='slave'} at 127.0.0.1:51086
I20260806 21:10:40.998032 15685 tablet_replica.cc:330] T 35c0a4590961409e8917357cd8a9f919 P 34bb2ff52a1142598fd3a989d5b94d11: stopping tablet replica
I20260806 21:10:40.998219 15308 tablet_service.cc:1514] Processing DeleteTablet for tablet 35c0a4590961409e8917357cd8a9f919 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:10:40 UTC) from {username='slave'} at 127.0.0.1:59350
I20260806 21:10:40.998376 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 15241
I20260806 21:10:40.998427 15685 raft_consensus.cc:2229] T 35c0a4590961409e8917357cd8a9f919 P 34bb2ff52a1142598fd3a989d5b94d11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:10:40.998894 15685 raft_consensus.cc:2258] T 35c0a4590961409e8917357cd8a9f919 P 34bb2ff52a1142598fd3a989d5b94d11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:10:41.000666 15685 ts_tablet_manager.cc:1905] T 35c0a4590961409e8917357cd8a9f919 P 34bb2ff52a1142598fd3a989d5b94d11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:10:41.003819 15685 ts_tablet_manager.cc:1918] T 35c0a4590961409e8917357cd8a9f919 P 34bb2ff52a1142598fd3a989d5b94d11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:10:41.003942 15685 log.cc:1196] T 35c0a4590961409e8917357cd8a9f919 P 34bb2ff52a1142598fd3a989d5b94d11: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-1/wal/wals/35c0a4590961409e8917357cd8a9f919
I20260806 21:10:41.004287 15685 ts_tablet_manager.cc:1939] T 35c0a4590961409e8917357cd8a9f919 P 34bb2ff52a1142598fd3a989d5b94d11: Deleting consensus metadata
I20260806 21:10:41.004704 15686 tablet_replica.cc:330] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b: stopping tablet replica
I20260806 21:10:41.005066 15686 raft_consensus.cc:2229] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:10:41.005517 15686 raft_consensus.cc:2258] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:10:41.006294 15163 catalog_manager.cc:4841] TS 34bb2ff52a1142598fd3a989d5b94d11 (127.27.225.194:43019): tablet 35c0a4590961409e8917357cd8a9f919 (table test-table [id=4333cdec07254958a84650f7ddf780df]) successfully deleted
I20260806 21:10:41.007272 15686 ts_tablet_manager.cc:1905] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 21:10:41.008083 15165 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.225.193:37061 (error 108)
I20260806 21:10:41.008931 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 15379
I20260806 21:10:41.011137 15657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:41.009329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:37061 (local address 127.0.0.1:59366)
0806 21:10:41.010895 (+ 1566us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:41.011001 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.193:37061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1263}
I20260806 21:10:41.011510 15686 ts_tablet_manager.cc:1918] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:10:41.011619 15686 log.cc:1196] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/35c0a4590961409e8917357cd8a9f919
I20260806 21:10:41.011967 15686 ts_tablet_manager.cc:1939] T 35c0a4590961409e8917357cd8a9f919 P ce781f06852c4fbca814789d0580be0b: Deleting consensus metadata
W20260806 21:10:41.012326 15165 catalog_manager.cc:4584] TS 32a3a545fa7e42318a50fe42110d50f4 (127.27.225.193:37061): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 35c0a4590961409e8917357cd8a9f919: Network error: Client connection negotiation failed: client connection to 127.27.225.193:37061: connect: Connection refused (error 111)
I20260806 21:10:41.013757 15164 catalog_manager.cc:4841] TS ce781f06852c4fbca814789d0580be0b (127.27.225.195:45925): tablet 35c0a4590961409e8917357cd8a9f919 (table test-table [id=4333cdec07254958a84650f7ddf780df]) successfully deleted
I20260806 21:10:41.017464 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 15514
I20260806 21:10:41.024391 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 15146
2026-08-06T21:10:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1180 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:10:41 dist-test-slave-69rg krb5kdc[15690](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:10:41 dist-test-slave-69rg krb5kdc[15690](info): set up 2 sockets
Aug 06 21:10:41 dist-test-slave-69rg krb5kdc[15690](info): commencing operation
krb5kdc: starting...
W20260806 21:10:42.135416 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:10:42 dist-test-slave-69rg krb5kdc[15690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:10:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:10:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:10:42.282488 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:34091
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:39769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:34091
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:10:42.400621 15703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:42.400928 15703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:42.400974 15703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:42.405238 15703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:10:42.405308 15703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:42.405328 15703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:42.405344 15703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:10:42.405359 15703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:10:42.410993 15703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:39769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:34091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:34091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:42.412155 15703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:42.413228 15703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:10:42.422255 15703 server_base.cc:1034] running on GCE node
W20260806 21:10:42.422200 15711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:42.422200 15708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:42.422199 15709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:42.423111 15703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:42.424327 15703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:42.425549 15703 hybrid_clock.cc:648] HybridClock initialized: now 1786050642425529 us; error 34 us; skew 500 ppm
Aug 06 21:10:42 dist-test-slave-69rg krb5kdc[15690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050642, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:42.433323 15703 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:10:42.434502 15703 webserver.cc:460] Webserver started at http://127.27.225.254:45849/ using document root <none> and password file <none>
I20260806 21:10:42.435101 15703 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:42.435205 15703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:42.435451 15703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:42.437613 15703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "d086c3535bf843edbcb353e74d2eb545"
format_stamp: "Formatted at 2026-08-06 21:10:42 on dist-test-slave-69rg"
server_key: "6064655f6f095aea84c9a11781c9ad34"
server_key_iv: "d2d3db9f0d2041cb4f1c8d9d709ce67e"
server_key_version: "encryptionkey@0"
I20260806 21:10:42.438289 15703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "d086c3535bf843edbcb353e74d2eb545"
format_stamp: "Formatted at 2026-08-06 21:10:42 on dist-test-slave-69rg"
server_key: "6064655f6f095aea84c9a11781c9ad34"
server_key_iv: "d2d3db9f0d2041cb4f1c8d9d709ce67e"
server_key_version: "encryptionkey@0"
I20260806 21:10:42.442178 15703 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:10:42.444561 15718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:42.445946 15703 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:10:42.446094 15703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "d086c3535bf843edbcb353e74d2eb545"
format_stamp: "Formatted at 2026-08-06 21:10:42 on dist-test-slave-69rg"
server_key: "6064655f6f095aea84c9a11781c9ad34"
server_key_iv: "d2d3db9f0d2041cb4f1c8d9d709ce67e"
server_key_version: "encryptionkey@0"
I20260806 21:10:42.446225 15703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:42.473658 15703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:42.476749 15703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:42.476967 15703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:42.485042 15703 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:34091
I20260806 21:10:42.485067 15772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:34091 every 8 connection(s)
I20260806 21:10:42.486153 15703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:10:42.488945 15773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:42.490828 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 15703
I20260806 21:10:42.490975 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:10:42.491297 28551 external_mini_cluster.cc:1428] Setting key 4a4e4f75452370c0aee38b3dabe3871e
I20260806 21:10:42.495944 15773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d086c3535bf843edbcb353e74d2eb545: Bootstrap starting.
I20260806 21:10:42.498661 15773 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d086c3535bf843edbcb353e74d2eb545: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:42.500017 15773 log.cc:824] T 00000000000000000000000000000000 P d086c3535bf843edbcb353e74d2eb545: Log is configured to *not* fsync() on all Append() calls
Aug 06 21:10:42 dist-test-slave-69rg krb5kdc[15690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:10:42.502269 15773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d086c3535bf843edbcb353e74d2eb545: No bootstrap required, opened a new log
I20260806 21:10:42.505946 15773 raft_consensus.cc:348] T 00000000000000000000000000000000 P d086c3535bf843edbcb353e74d2eb545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d086c3535bf843edbcb353e74d2eb545" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 34091 } }
I20260806 21:10:42.506217 15773 raft_consensus.cc:374] T 00000000000000000000000000000000 P d086c3535bf843edbcb353e74d2eb545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:42.506299 15773 raft_consensus.cc:729] T 00000000000000000000000000000000 P d086c3535bf843edbcb353e74d2eb545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d086c3535bf843edbcb353e74d2eb545, State: Initialized, Role: FOLLOWER
I20260806 21:10:42.506743 15776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:42.492789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52878 (local address 127.27.225.254:34091)
0806 21:10:42.493246 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:42.493265 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:42.493301 (+ 36us) server_negotiation.cc:407] Connection header received
0806 21:10:42.494035 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:42.494054 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:42.494390 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:42.494667 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:42.495615 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:42.496680 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:42.497913 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:42.498399 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:42.498528 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:42.501693 (+ 3165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:42.501698 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:42.501712 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:10:42.501740 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:42.503678 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:42.504163 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:42.504168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:42.504173 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:42.504288 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:42.504592 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:42.504595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:42.504597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:42.505004 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:42.505249 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:42.505507 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 21:10:42.505874 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":157,"threads_started":1}
I20260806 21:10:42.506819 15773 consensus_queue.cc:260] T 00000000000000000000000000000000 P d086c3535bf843edbcb353e74d2eb545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d086c3535bf843edbcb353e74d2eb545" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 34091 } }
I20260806 21:10:42.507026 15773 raft_consensus.cc:388] T 00000000000000000000000000000000 P d086c3535bf843edbcb353e74d2eb545 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:10:42.507117 15773 raft_consensus.cc:482] T 00000000000000000000000000000000 P d086c3535bf843edbcb353e74d2eb545 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:10:42.507218 15773 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d086c3535bf843edbcb353e74d2eb545 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:42.508208 15773 raft_consensus.cc:504] T 00000000000000000000000000000000 P d086c3535bf843edbcb353e74d2eb545 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d086c3535bf843edbcb353e74d2eb545" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 34091 } }
I20260806 21:10:42.508569 15773 leader_election.cc:302] T 00000000000000000000000000000000 P d086c3535bf843edbcb353e74d2eb545 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d086c3535bf843edbcb353e74d2eb545; no voters:
I20260806 21:10:42.508929 15773 leader_election.cc:288] T 00000000000000000000000000000000 P d086c3535bf843edbcb353e74d2eb545 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:10:42.509014 15778 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d086c3535bf843edbcb353e74d2eb545 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:10:42.509315 15778 raft_consensus.cc:686] T 00000000000000000000000000000000 P d086c3535bf843edbcb353e74d2eb545 [term 1 LEADER]: Becoming Leader. State: Replica: d086c3535bf843edbcb353e74d2eb545, State: Running, Role: LEADER
I20260806 21:10:42.509624 15778 consensus_queue.cc:237] T 00000000000000000000000000000000 P d086c3535bf843edbcb353e74d2eb545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d086c3535bf843edbcb353e74d2eb545" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 34091 } }
I20260806 21:10:42.510401 15773 sys_catalog.cc:564] T 00000000000000000000000000000000 P d086c3535bf843edbcb353e74d2eb545 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:10:42.511476 15779 sys_catalog.cc:455] T 00000000000000000000000000000000 P d086c3535bf843edbcb353e74d2eb545 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d086c3535bf843edbcb353e74d2eb545. Latest consensus state: current_term: 1 leader_uuid: "d086c3535bf843edbcb353e74d2eb545" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d086c3535bf843edbcb353e74d2eb545" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 34091 } } }
I20260806 21:10:42.511510 15780 sys_catalog.cc:455] T 00000000000000000000000000000000 P d086c3535bf843edbcb353e74d2eb545 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d086c3535bf843edbcb353e74d2eb545" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d086c3535bf843edbcb353e74d2eb545" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 34091 } } }
I20260806 21:10:42.511732 15780 sys_catalog.cc:458] T 00000000000000000000000000000000 P d086c3535bf843edbcb353e74d2eb545 [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:42.512046 15779 sys_catalog.cc:458] T 00000000000000000000000000000000 P d086c3535bf843edbcb353e74d2eb545 [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:42.512177 15787 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:10:42.515588 15787 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:10:42.521554 15787 catalog_manager.cc:1347] Generated new cluster ID: 409e73e3faf144fd932cfca426809c2e
I20260806 21:10:42.521646 15787 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:10:42.540795 15787 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:10:42.542738 15787 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:10:42.558743 15787 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d086c3535bf843edbcb353e74d2eb545: Generated new TSK 0
I20260806 21:10:42.559499 15787 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.225.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
Entry for principal kudu/127.27.225.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab.
I20260806 21:10:42.593730 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:34091
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:39769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:10:42.714954 15799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:42.715216 15799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:42.715258 15799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:42.719295 15799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:42.719359 15799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:42.719460 15799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:10:42.724936 15799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:39769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.193.keytab
--principal=kudu/127.27.225.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:34091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:42.726150 15799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:42.727244 15799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:42.737576 15805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:42.737592 15804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:42.737643 15807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:42.737779 15799 server_base.cc:1034] running on GCE node
I20260806 21:10:42.738410 15799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:42.739091 15799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:42.740338 15799 hybrid_clock.cc:648] HybridClock initialized: now 1786050642740322 us; error 28 us; skew 500 ppm
Aug 06 21:10:42 dist-test-slave-69rg krb5kdc[15690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050642, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:42.743484 15799 init.cc:377] Logged in from keytab as kudu/127.27.225.193@KRBTEST.COM (short username kudu)
I20260806 21:10:42.744421 15799 webserver.cc:460] Webserver started at http://127.27.225.193:37051/ using document root <none> and password file <none>
I20260806 21:10:42.744999 15799 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:42.745092 15799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:42.745340 15799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:42.747537 15799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "80cf00d2cc7e46bfbcfec726a5699d99"
format_stamp: "Formatted at 2026-08-06 21:10:42 on dist-test-slave-69rg"
server_key: "36257c0a7e5e08fe7c3b389510e42ef5"
server_key_iv: "3ee331cbcdc1b78132ea72677d797a63"
server_key_version: "encryptionkey@0"
I20260806 21:10:42.748147 15799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "80cf00d2cc7e46bfbcfec726a5699d99"
format_stamp: "Formatted at 2026-08-06 21:10:42 on dist-test-slave-69rg"
server_key: "36257c0a7e5e08fe7c3b389510e42ef5"
server_key_iv: "3ee331cbcdc1b78132ea72677d797a63"
server_key_version: "encryptionkey@0"
I20260806 21:10:42.752045 15799 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:10:42.754594 15814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:42.755936 15799 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:10:42.756083 15799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "80cf00d2cc7e46bfbcfec726a5699d99"
format_stamp: "Formatted at 2026-08-06 21:10:42 on dist-test-slave-69rg"
server_key: "36257c0a7e5e08fe7c3b389510e42ef5"
server_key_iv: "3ee331cbcdc1b78132ea72677d797a63"
server_key_version: "encryptionkey@0"
I20260806 21:10:42.756206 15799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:42.785210 15799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:42.788754 15799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:42.789037 15799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:42.789722 15799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:10:42.790673 15799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:10:42.790747 15799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:42.790820 15799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:10:42.791033 15799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:42.800460 15799 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:40381
I20260806 21:10:42.800485 15929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:40381 every 8 connection(s)
I20260806 21:10:42.801409 15799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:10:42.802345 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 15799
I20260806 21:10:42.802474 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:10:42.802781 28551 external_mini_cluster.cc:1428] Setting key 1c0f5620547422d4561112bf3ace04df
Aug 06 21:10:42 dist-test-slave-69rg krb5kdc[15690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050642, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.193@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.225.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.194@KRBTEST.COM" created.
I20260806 21:10:42.818538 15776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:42.803622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:39141 (local address 127.27.225.254:34091)
0806 21:10:42.803781 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:42.803787 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:42.804667 (+ 880us) server_negotiation.cc:407] Connection header received
0806 21:10:42.806097 (+ 1430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:42.806103 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:42.806170 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:42.806272 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:42.808141 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:42.809004 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:42.810372 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:42.810835 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:42.811000 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:42.814185 (+ 3185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:42.814188 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:42.814190 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:10:42.814221 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:42.815931 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:42.816562 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:42.816565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:42.816566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:42.816653 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:42.817045 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:42.817048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:42.817049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:42.817266 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:42.817349 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:42.818090 (+ 741us) server_negotiation.cc:299] Negotiation successful
0806 21:10:42.818303 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 21:10:42.819453 15932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:42.804004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:34091 (local address 127.27.225.193:39141)
0806 21:10:42.804501 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:42.804554 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:42.805802 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:42.806546 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:42.806556 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:42.807135 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:10:42.807901 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:42.807926 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:42.809196 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:42.809198 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:42.810204 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:42.810215 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:42.811047 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:42.811050 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:42.811291 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:42.812040 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:10:42.812069 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:10:42.813977 (+ 1908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:10:42.816063 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:10:42.816068 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:10:42.816084 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:10:42.816438 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:10:42.816769 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:10:42.816771 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:10:42.816773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:10:42.816939 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:10:42.817385 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:10:42.817391 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:10:42.817751 (+ 360us) client_negotiation.cc:769] Sending connection context
0806 21:10:42.818015 (+ 264us) client_negotiation.cc:240] Negotiation successful
0806 21:10:42.818366 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":147,"threads_started":1}
I20260806 21:10:42.820890 15930 heartbeater.cc:347] Connected to a master server at 127.27.225.254:34091
I20260806 21:10:42.821187 15930 heartbeater.cc:464] Registering TS with master...
I20260806 21:10:42.821762 15930 heartbeater.cc:511] Master 127.27.225.254:34091 requested a full tablet report, sending...
I20260806 21:10:42.823402 15735 ts_manager.cc:194] Registered new tserver with Master: 80cf00d2cc7e46bfbcfec726a5699d99 (127.27.225.193:40381)
I20260806 21:10:42.824587 15735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.193@KRBTEST.COM'} at 127.27.225.193:39141
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
Entry for principal kudu/127.27.225.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab.
I20260806 21:10:42.837997 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:34091
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:39769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:10:42.959481 15935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:42.959764 15935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:42.959860 15935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:42.964159 15935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:42.964236 15935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:42.964298 15935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:10:42.969844 15935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:39769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.194.keytab
--principal=kudu/127.27.225.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:34091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:42.970957 15935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:42.971963 15935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:42.982215 15941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:42.982333 15943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:42.982381 15940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:42.982573 15935 server_base.cc:1034] running on GCE node
I20260806 21:10:42.982996 15935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:42.983722 15935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:42.984915 15935 hybrid_clock.cc:648] HybridClock initialized: now 1786050642984877 us; error 47 us; skew 500 ppm
Aug 06 21:10:42 dist-test-slave-69rg krb5kdc[15690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050642, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:42.988075 15935 init.cc:377] Logged in from keytab as kudu/127.27.225.194@KRBTEST.COM (short username kudu)
I20260806 21:10:42.988988 15935 webserver.cc:460] Webserver started at http://127.27.225.194:37225/ using document root <none> and password file <none>
I20260806 21:10:42.989567 15935 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:42.989632 15935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:42.989957 15935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:42.992082 15935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "d37409d5e8b9498daa2a53ffa00aa085"
format_stamp: "Formatted at 2026-08-06 21:10:42 on dist-test-slave-69rg"
server_key: "afcf5839dd682b015c47dc3bf68d6ebf"
server_key_iv: "c1796f26ade2440b612463719ef9ee70"
server_key_version: "encryptionkey@0"
I20260806 21:10:42.992678 15935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "d37409d5e8b9498daa2a53ffa00aa085"
format_stamp: "Formatted at 2026-08-06 21:10:42 on dist-test-slave-69rg"
server_key: "afcf5839dd682b015c47dc3bf68d6ebf"
server_key_iv: "c1796f26ade2440b612463719ef9ee70"
server_key_version: "encryptionkey@0"
I20260806 21:10:42.996670 15935 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:10:42.999364 15952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:43.000751 15935 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 21:10:43.000916 15935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "d37409d5e8b9498daa2a53ffa00aa085"
format_stamp: "Formatted at 2026-08-06 21:10:42 on dist-test-slave-69rg"
server_key: "afcf5839dd682b015c47dc3bf68d6ebf"
server_key_iv: "c1796f26ade2440b612463719ef9ee70"
server_key_version: "encryptionkey@0"
I20260806 21:10:43.001050 15935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:43.018042 15935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:43.022563 15935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:43.022840 15935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:43.023528 15935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:10:43.024472 15935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:10:43.024524 15935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:43.024566 15935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:10:43.024608 15935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:43.034089 15935 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:44843
I20260806 21:10:43.034222 16069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:44843 every 8 connection(s)
I20260806 21:10:43.035080 15935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:10:43.036667 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 15935
I20260806 21:10:43.036816 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:10:43.037170 28551 external_mini_cluster.cc:1428] Setting key 85e57213f742012b766df611dca74495
Aug 06 21:10:43 dist-test-slave-69rg krb5kdc[15690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050642, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.194@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.225.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.195@KRBTEST.COM" created.
I20260806 21:10:43.050951 15776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:43.037105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:48335 (local address 127.27.225.254:34091)
0806 21:10:43.037266 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:43.037272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:43.038433 (+ 1161us) server_negotiation.cc:407] Connection header received
0806 21:10:43.039868 (+ 1435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:43.039872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:43.039932 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:43.040010 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:43.041225 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.041873 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.043165 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.043410 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.043528 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:43.046544 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:43.046547 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:43.046549 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:10:43.046577 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:43.048421 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:43.049006 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:43.049011 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:43.049012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:43.049113 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:43.049523 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:43.049529 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:43.049533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:43.049745 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:43.049902 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:43.050508 (+ 606us) server_negotiation.cc:299] Negotiation successful
0806 21:10:43.050672 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 21:10:43.051965 16072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:43.037609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:34091 (local address 127.27.225.194:48335)
0806 21:10:43.038232 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:43.038291 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:43.039633 (+ 1342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:43.040185 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:43.040192 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:43.040575 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:10:43.041065 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:43.041078 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.042015 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.042023 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:43.043027 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:43.043038 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.043576 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.043581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:43.043766 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:43.044420 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:10:43.044437 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:10:43.046326 (+ 1889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:10:43.048565 (+ 2239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:10:43.048573 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:10:43.048592 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:10:43.048892 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:10:43.049232 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:10:43.049237 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:10:43.049242 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:10:43.049420 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:10:43.049901 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:10:43.049909 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:10:43.050202 (+ 293us) client_negotiation.cc:769] Sending connection context
0806 21:10:43.050419 (+ 217us) client_negotiation.cc:240] Negotiation successful
0806 21:10:43.050836 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":190,"threads_started":1}
I20260806 21:10:43.053606 16070 heartbeater.cc:347] Connected to a master server at 127.27.225.254:34091
I20260806 21:10:43.053925 16070 heartbeater.cc:464] Registering TS with master...
I20260806 21:10:43.054466 16070 heartbeater.cc:511] Master 127.27.225.254:34091 requested a full tablet report, sending...
I20260806 21:10:43.055662 15735 ts_manager.cc:194] Registered new tserver with Master: d37409d5e8b9498daa2a53ffa00aa085 (127.27.225.194:44843)
I20260806 21:10:43.056293 15735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.194@KRBTEST.COM'} at 127.27.225.194:48335
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
Entry for principal kudu/127.27.225.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab.
I20260806 21:10:43.073045 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:34091
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:39769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:10:43.191319 16076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:43.191598 16076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:43.191643 16076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:43.195817 16076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:43.195904 16076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:43.195966 16076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:10:43.201865 16076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:39769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.195.keytab
--principal=kudu/127.27.225.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:34091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:43.203436 16076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:43.205093 16076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:43.215780 16082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:43.215771 16084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:43.215785 16081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:43.216040 16076 server_base.cc:1034] running on GCE node
I20260806 21:10:43.216602 16076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:43.217325 16076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:43.218542 16076 hybrid_clock.cc:648] HybridClock initialized: now 1786050643218528 us; error 26 us; skew 500 ppm
Aug 06 21:10:43 dist-test-slave-69rg krb5kdc[15690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050643, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:43.221464 16076 init.cc:377] Logged in from keytab as kudu/127.27.225.195@KRBTEST.COM (short username kudu)
I20260806 21:10:43.222503 16076 webserver.cc:460] Webserver started at http://127.27.225.195:37641/ using document root <none> and password file <none>
I20260806 21:10:43.223047 16076 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:43.223106 16076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:43.223276 16076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:43.225507 16076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "9ae85a8e675e43c1835c72aac7313b61"
format_stamp: "Formatted at 2026-08-06 21:10:43 on dist-test-slave-69rg"
server_key: "ec80a8b2482a52b9d5648403b64aa880"
server_key_iv: "934fccf7e0d18a54a9aefb2bc800692e"
server_key_version: "encryptionkey@0"
I20260806 21:10:43.226150 16076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "9ae85a8e675e43c1835c72aac7313b61"
format_stamp: "Formatted at 2026-08-06 21:10:43 on dist-test-slave-69rg"
server_key: "ec80a8b2482a52b9d5648403b64aa880"
server_key_iv: "934fccf7e0d18a54a9aefb2bc800692e"
server_key_version: "encryptionkey@0"
I20260806 21:10:43.230116 16076 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:10:43.232813 16091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:43.234303 16076 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:10:43.234450 16076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "9ae85a8e675e43c1835c72aac7313b61"
format_stamp: "Formatted at 2026-08-06 21:10:43 on dist-test-slave-69rg"
server_key: "ec80a8b2482a52b9d5648403b64aa880"
server_key_iv: "934fccf7e0d18a54a9aefb2bc800692e"
server_key_version: "encryptionkey@0"
I20260806 21:10:43.234578 16076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:43.265129 16076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:43.268785 16076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:43.269018 16076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:43.269711 16076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:10:43.270748 16076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:10:43.270798 16076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:43.270839 16076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:10:43.270855 16076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:43.286191 16076 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:45671
I20260806 21:10:43.286314 16208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:45671 every 8 connection(s)
I20260806 21:10:43.287214 16076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:10:43.291934 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 16076
I20260806 21:10:43.292197 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:10:43.292567 28551 external_mini_cluster.cc:1428] Setting key c6aa829862007893ff4eae299c6082aa
Aug 06 21:10:43 dist-test-slave-69rg krb5kdc[15690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050643, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.195@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:10:43.306679 15776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:43.289340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:59425 (local address 127.27.225.254:34091)
0806 21:10:43.289516 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:43.289521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:43.290582 (+ 1061us) server_negotiation.cc:407] Connection header received
0806 21:10:43.292615 (+ 2033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:43.292619 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:43.292688 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:43.292802 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:43.294630 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.295374 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.297555 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.297894 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.298004 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:43.301567 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:43.301571 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:43.301574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:10:43.301608 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:43.303851 (+ 2243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:43.304648 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:43.304652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:43.304653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:43.304769 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:43.305204 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:43.305207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:43.305208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:43.305452 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:43.305541 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:43.306329 (+ 788us) server_negotiation.cc:299] Negotiation successful
0806 21:10:43.306488 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 21:10:43.308274 16211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:43.289743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:34091 (local address 127.27.225.195:59425)
0806 21:10:43.290381 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:43.290445 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:43.292328 (+ 1883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:43.293042 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:43.293054 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:43.293617 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:10:43.294449 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:43.294474 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.295570 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.295576 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:43.297380 (+ 1804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:43.297398 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.298098 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.298104 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:43.298366 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:43.299161 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:10:43.299191 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:10:43.301314 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:10:43.304013 (+ 2699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:10:43.304023 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:10:43.304046 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:10:43.304474 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:10:43.304870 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:10:43.304875 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:10:43.304880 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:10:43.305090 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:10:43.305570 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:10:43.305578 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:10:43.305998 (+ 420us) client_negotiation.cc:769] Sending connection context
0806 21:10:43.306233 (+ 235us) client_negotiation.cc:240] Negotiation successful
0806 21:10:43.306827 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":138,"threads_started":1}
I20260806 21:10:43.309590 16209 heartbeater.cc:347] Connected to a master server at 127.27.225.254:34091
I20260806 21:10:43.309899 16209 heartbeater.cc:464] Registering TS with master...
I20260806 21:10:43.310609 16209 heartbeater.cc:511] Master 127.27.225.254:34091 requested a full tablet report, sending...
I20260806 21:10:43.312026 15735 ts_manager.cc:194] Registered new tserver with Master: 9ae85a8e675e43c1835c72aac7313b61 (127.27.225.195:45671)
I20260806 21:10:43.312740 15735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.225.195@KRBTEST.COM'} at 127.27.225.195:59425
I20260806 21:10:43.319864 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:10:43.330461 15776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:43.322071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52884 (local address 127.27.225.254:34091)
0806 21:10:43.322222 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:43.322226 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:43.322492 (+ 266us) server_negotiation.cc:407] Connection header received
0806 21:10:43.322668 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:43.322671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:43.322730 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:43.322911 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:43.323704 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.324248 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.325116 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.325362 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.325477 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:43.327026 (+ 1549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:43.327030 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:43.327033 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:10:43.327071 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:43.328590 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:43.329053 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:43.329056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:43.329058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:43.329147 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:43.329446 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:43.329450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:43.329452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:43.329703 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:43.329921 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:43.330159 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 21:10:43.330283 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 21:10:43.334556 15735 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:10:43.337728 15735 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:10:43.355746 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:43.347506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:44843 (local address 127.0.0.1:39744)
0806 21:10:43.348532 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:43.348591 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:43.348726 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:43.350511 (+ 1785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:43.350514 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:43.350528 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:43.350784 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:43.350792 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.352370 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.352373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:43.353965 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:43.353977 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.355171 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.355175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:43.355292 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:43.355345 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 21:10:43.355444 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 21:10:43.355578 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"mutex_wait_us":156,"thread_start_us":91,"threads_started":1}
I20260806 21:10:43.356520 16221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:43.347516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:45671 (local address 127.0.0.1:55098)
0806 21:10:43.349374 (+ 1858us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:43.349393 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:43.349581 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:43.350400 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:43.350408 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:43.350439 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:43.350786 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:43.350792 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.353239 (+ 2447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.353242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:43.354466 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:43.354476 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.356085 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.356088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:43.356195 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:43.356222 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 21:10:43.356289 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 21:10:43.356374 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1765,"mutex_wait_us":119,"spinlock_wait_cycles":1792,"thread_start_us":152,"threads_started":1}
I20260806 21:10:43.356527 16224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:43.348522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43702 (local address 127.27.225.193:40381)
0806 21:10:43.349025 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:43.349035 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:43.349534 (+ 499us) server_negotiation.cc:407] Connection header received
0806 21:10:43.349853 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:43.349859 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:43.350080 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:43.350244 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:43.350963 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.352352 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.354373 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.355437 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.355557 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:43.355717 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:43.356183 (+ 466us) server_negotiation.cc:299] Negotiation successful
0806 21:10:43.356321 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":96,"threads_started":1}
I20260806 21:10:43.356978 16225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:43.348887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55098 (local address 127.27.225.195:45671)
0806 21:10:43.349291 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:43.349305 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:43.349484 (+ 179us) server_negotiation.cc:407] Connection header received
0806 21:10:43.349747 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:43.349754 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:43.350014 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:43.350173 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:43.350926 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.352198 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.354610 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.355460 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.355546 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:43.355988 (+ 442us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:43.356693 (+ 705us) server_negotiation.cc:299] Negotiation successful
0806 21:10:43.356820 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":152,"threads_started":1}
I20260806 21:10:43.357793 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:43.348360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39744 (local address 127.27.225.194:44843)
0806 21:10:43.349981 (+ 1621us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:43.349994 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:43.350016 (+ 22us) server_negotiation.cc:407] Connection header received
0806 21:10:43.350091 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:43.350099 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:43.350362 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:43.350887 (+ 525us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:43.351055 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.352198 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.354107 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.355024 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.357416 (+ 2392us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:43.357558 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:43.357600 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 21:10:43.357676 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1518,"thread_start_us":141,"threads_started":1}
I20260806 21:10:43.358011 16222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:43.347762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:40381 (local address 127.0.0.1:43702)
0806 21:10:43.349439 (+ 1677us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:43.349453 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:43.349615 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:43.350476 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:43.350481 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:43.350518 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:43.350799 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:43.350809 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.352522 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.352527 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:43.354208 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:43.354223 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.355833 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.355838 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:43.355983 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:43.356018 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 21:10:43.356871 (+ 853us) client_negotiation.cc:240] Negotiation successful
0806 21:10:43.356978 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1611,"mutex_wait_us":88,"spinlock_wait_cycles":768,"thread_start_us":112,"threads_started":1}
I20260806 21:10:43.360915 16002 tablet_service.cc:1467] Processing CreateTablet for tablet 1ef576fd7e254706acafaed0ab024d86 (DEFAULT_TABLE table=test-table [id=5272cc13d63e4e4dafb2794c64a0ac56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:10:43.360913 15863 tablet_service.cc:1467] Processing CreateTablet for tablet 1ef576fd7e254706acafaed0ab024d86 (DEFAULT_TABLE table=test-table [id=5272cc13d63e4e4dafb2794c64a0ac56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:10:43.360720 16139 tablet_service.cc:1467] Processing CreateTablet for tablet 1ef576fd7e254706acafaed0ab024d86 (DEFAULT_TABLE table=test-table [id=5272cc13d63e4e4dafb2794c64a0ac56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:10:43.362051 15863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ef576fd7e254706acafaed0ab024d86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:43.362051 16002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ef576fd7e254706acafaed0ab024d86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:43.362077 16139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ef576fd7e254706acafaed0ab024d86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:43.371873 16227 tablet_bootstrap.cc:492] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61: Bootstrap starting.
I20260806 21:10:43.372357 16226 tablet_bootstrap.cc:492] T 1ef576fd7e254706acafaed0ab024d86 P 80cf00d2cc7e46bfbcfec726a5699d99: Bootstrap starting.
I20260806 21:10:43.374176 16228 tablet_bootstrap.cc:492] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085: Bootstrap starting.
I20260806 21:10:43.374393 16226 tablet_bootstrap.cc:654] T 1ef576fd7e254706acafaed0ab024d86 P 80cf00d2cc7e46bfbcfec726a5699d99: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:43.374550 16227 tablet_bootstrap.cc:654] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:43.375324 16226 log.cc:824] T 1ef576fd7e254706acafaed0ab024d86 P 80cf00d2cc7e46bfbcfec726a5699d99: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:43.375901 16227 log.cc:824] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:43.376689 16228 tablet_bootstrap.cc:654] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:43.377638 16226 tablet_bootstrap.cc:492] T 1ef576fd7e254706acafaed0ab024d86 P 80cf00d2cc7e46bfbcfec726a5699d99: No bootstrap required, opened a new log
I20260806 21:10:43.377887 16226 ts_tablet_manager.cc:1397] T 1ef576fd7e254706acafaed0ab024d86 P 80cf00d2cc7e46bfbcfec726a5699d99: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 21:10:43.378098 16227 tablet_bootstrap.cc:492] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61: No bootstrap required, opened a new log
I20260806 21:10:43.378296 16227 ts_tablet_manager.cc:1397] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 21:10:43.379495 16228 log.cc:824] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:43.382066 16226 raft_consensus.cc:348] T 1ef576fd7e254706acafaed0ab024d86 P 80cf00d2cc7e46bfbcfec726a5699d99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae85a8e675e43c1835c72aac7313b61" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 45671 } } peers { permanent_uuid: "d37409d5e8b9498daa2a53ffa00aa085" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 44843 } } peers { permanent_uuid: "80cf00d2cc7e46bfbcfec726a5699d99" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40381 } }
I20260806 21:10:43.382349 16226 raft_consensus.cc:374] T 1ef576fd7e254706acafaed0ab024d86 P 80cf00d2cc7e46bfbcfec726a5699d99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:43.382404 16226 raft_consensus.cc:729] T 1ef576fd7e254706acafaed0ab024d86 P 80cf00d2cc7e46bfbcfec726a5699d99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80cf00d2cc7e46bfbcfec726a5699d99, State: Initialized, Role: FOLLOWER
I20260806 21:10:43.382862 16226 consensus_queue.cc:260] T 1ef576fd7e254706acafaed0ab024d86 P 80cf00d2cc7e46bfbcfec726a5699d99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae85a8e675e43c1835c72aac7313b61" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 45671 } } peers { permanent_uuid: "d37409d5e8b9498daa2a53ffa00aa085" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 44843 } } peers { permanent_uuid: "80cf00d2cc7e46bfbcfec726a5699d99" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40381 } }
I20260806 21:10:43.383601 15930 heartbeater.cc:503] Master 127.27.225.254:34091 was elected leader, sending a full tablet report...
I20260806 21:10:43.383929 16226 ts_tablet_manager.cc:1428] T 1ef576fd7e254706acafaed0ab024d86 P 80cf00d2cc7e46bfbcfec726a5699d99: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 21:10:43.383828 16227 raft_consensus.cc:348] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae85a8e675e43c1835c72aac7313b61" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 45671 } } peers { permanent_uuid: "d37409d5e8b9498daa2a53ffa00aa085" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 44843 } } peers { permanent_uuid: "80cf00d2cc7e46bfbcfec726a5699d99" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40381 } }
I20260806 21:10:43.384145 16227 raft_consensus.cc:374] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:43.384209 16227 raft_consensus.cc:729] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ae85a8e675e43c1835c72aac7313b61, State: Initialized, Role: FOLLOWER
I20260806 21:10:43.384675 16227 consensus_queue.cc:260] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae85a8e675e43c1835c72aac7313b61" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 45671 } } peers { permanent_uuid: "d37409d5e8b9498daa2a53ffa00aa085" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 44843 } } peers { permanent_uuid: "80cf00d2cc7e46bfbcfec726a5699d99" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40381 } }
I20260806 21:10:43.385763 16227 ts_tablet_manager.cc:1428] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 21:10:43.386102 16228 tablet_bootstrap.cc:492] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085: No bootstrap required, opened a new log
I20260806 21:10:43.386211 16209 heartbeater.cc:503] Master 127.27.225.254:34091 was elected leader, sending a full tablet report...
I20260806 21:10:43.386300 16228 ts_tablet_manager.cc:1397] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.000s
I20260806 21:10:43.390494 16228 raft_consensus.cc:348] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae85a8e675e43c1835c72aac7313b61" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 45671 } } peers { permanent_uuid: "d37409d5e8b9498daa2a53ffa00aa085" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 44843 } } peers { permanent_uuid: "80cf00d2cc7e46bfbcfec726a5699d99" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40381 } }
I20260806 21:10:43.390754 16228 raft_consensus.cc:374] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:43.390822 16228 raft_consensus.cc:729] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d37409d5e8b9498daa2a53ffa00aa085, State: Initialized, Role: FOLLOWER
I20260806 21:10:43.391233 16228 consensus_queue.cc:260] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae85a8e675e43c1835c72aac7313b61" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 45671 } } peers { permanent_uuid: "d37409d5e8b9498daa2a53ffa00aa085" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 44843 } } peers { permanent_uuid: "80cf00d2cc7e46bfbcfec726a5699d99" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40381 } }
I20260806 21:10:43.391969 16070 heartbeater.cc:503] Master 127.27.225.254:34091 was elected leader, sending a full tablet report...
I20260806 21:10:43.392300 16228 ts_tablet_manager.cc:1428] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260806 21:10:43.536315 16071 tablet.cc:2369] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 21:10:43.538209 16210 tablet.cc:2369] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 21:10:43.554064 15931 tablet.cc:2369] T 1ef576fd7e254706acafaed0ab024d86 P 80cf00d2cc7e46bfbcfec726a5699d99: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:10:43.585681 16233 raft_consensus.cc:482] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:10:43.586076 16233 raft_consensus.cc:504] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae85a8e675e43c1835c72aac7313b61" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 45671 } } peers { permanent_uuid: "d37409d5e8b9498daa2a53ffa00aa085" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 44843 } } peers { permanent_uuid: "80cf00d2cc7e46bfbcfec726a5699d99" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40381 } }
I20260806 21:10:43.587551 16233 leader_election.cc:288] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d37409d5e8b9498daa2a53ffa00aa085 (127.27.225.194:44843), 80cf00d2cc7e46bfbcfec726a5699d99 (127.27.225.193:40381)
I20260806 21:10:43.591327 16234 raft_consensus.cc:482] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:10:43.591526 16234 raft_consensus.cc:504] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae85a8e675e43c1835c72aac7313b61" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 45671 } } peers { permanent_uuid: "d37409d5e8b9498daa2a53ffa00aa085" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 44843 } } peers { permanent_uuid: "80cf00d2cc7e46bfbcfec726a5699d99" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40381 } }
I20260806 21:10:43.592285 16236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:43.587824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:44843 (local address 127.27.225.195:45303)
0806 21:10:43.588345 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:43.588357 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:43.588507 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:43.588780 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:43.588782 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:43.588798 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:43.588931 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:43.588936 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.589654 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.589667 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:43.590631 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:43.590644 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.591812 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.591815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:43.591912 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:43.591938 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 21:10:43.592018 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 21:10:43.592094 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"mutex_wait_us":182,"spinlock_wait_cycles":24704,"thread_start_us":117,"threads_started":1}
I20260806 21:10:43.592309 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:43.587946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:45303 (local address 127.27.225.194:44843)
0806 21:10:43.588171 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:43.588178 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:43.588445 (+ 267us) server_negotiation.cc:407] Connection header received
0806 21:10:43.588620 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:43.588629 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:43.588675 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:43.588762 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:43.589041 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.589529 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.590865 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.591528 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.591665 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:43.591720 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:43.592064 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 21:10:43.592175 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260806 21:10:43.592573 16224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:43.587975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:47663 (local address 127.27.225.193:40381)
0806 21:10:43.588127 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:43.588133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:43.588151 (+ 18us) server_negotiation.cc:407] Connection header received
0806 21:10:43.588294 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:43.588297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:43.588346 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:43.588444 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:43.588770 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.589332 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.590778 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.591528 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.591665 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:43.591720 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:43.592352 (+ 632us) server_negotiation.cc:299] Negotiation successful
0806 21:10:43.592448 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 21:10:43.592878 16211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:43.587825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:40381 (local address 127.27.225.195:47663)
0806 21:10:43.588004 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:43.588023 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:43.588139 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:43.588460 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:43.588463 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:43.588504 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:43.588633 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:43.588639 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.589464 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.589469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:43.590628 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:43.590639 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.591881 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.591885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:43.592035 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:43.592075 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 21:10:43.592153 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 21:10:43.592228 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"mutex_wait_us":2,"spinlock_wait_cycles":1920}
I20260806 21:10:43.593094 16022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ef576fd7e254706acafaed0ab024d86" candidate_uuid: "9ae85a8e675e43c1835c72aac7313b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d37409d5e8b9498daa2a53ffa00aa085" is_pre_election: true
I20260806 21:10:43.593358 16022 raft_consensus.cc:2454] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ae85a8e675e43c1835c72aac7313b61 in term 0.
I20260806 21:10:43.593924 16093 leader_election.cc:302] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ae85a8e675e43c1835c72aac7313b61, d37409d5e8b9498daa2a53ffa00aa085; no voters:
I20260806 21:10:43.594027 16234 leader_election.cc:288] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ae85a8e675e43c1835c72aac7313b61 (127.27.225.195:45671), 80cf00d2cc7e46bfbcfec726a5699d99 (127.27.225.193:40381)
I20260806 21:10:43.593972 15884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ef576fd7e254706acafaed0ab024d86" candidate_uuid: "9ae85a8e675e43c1835c72aac7313b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80cf00d2cc7e46bfbcfec726a5699d99" is_pre_election: true
I20260806 21:10:43.594286 15884 raft_consensus.cc:2454] T 1ef576fd7e254706acafaed0ab024d86 P 80cf00d2cc7e46bfbcfec726a5699d99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ae85a8e675e43c1835c72aac7313b61 in term 0.
I20260806 21:10:43.594319 16233 raft_consensus.cc:2781] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:10:43.594408 16233 raft_consensus.cc:482] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:10:43.594450 16233 raft_consensus.cc:3037] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:43.595835 16233 raft_consensus.cc:504] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae85a8e675e43c1835c72aac7313b61" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 45671 } } peers { permanent_uuid: "d37409d5e8b9498daa2a53ffa00aa085" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 44843 } } peers { permanent_uuid: "80cf00d2cc7e46bfbcfec726a5699d99" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40381 } }
I20260806 21:10:43.596194 16233 leader_election.cc:288] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61 [CANDIDATE]: Term 1 election: Requested vote from peers d37409d5e8b9498daa2a53ffa00aa085 (127.27.225.194:44843), 80cf00d2cc7e46bfbcfec726a5699d99 (127.27.225.193:40381)
I20260806 21:10:43.596534 16022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ef576fd7e254706acafaed0ab024d86" candidate_uuid: "9ae85a8e675e43c1835c72aac7313b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d37409d5e8b9498daa2a53ffa00aa085"
I20260806 21:10:43.596716 16022 raft_consensus.cc:3037] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:43.596719 15884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ef576fd7e254706acafaed0ab024d86" candidate_uuid: "9ae85a8e675e43c1835c72aac7313b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80cf00d2cc7e46bfbcfec726a5699d99"
I20260806 21:10:43.596864 15884 raft_consensus.cc:3037] T 1ef576fd7e254706acafaed0ab024d86 P 80cf00d2cc7e46bfbcfec726a5699d99 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:43.597795 16238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:43.593036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:45671 (local address 127.27.225.194:58771)
0806 21:10:43.593305 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:43.593323 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:43.593421 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:43.593874 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:43.593878 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:43.593898 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:43.594016 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:43.594023 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.595088 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.595091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:43.596319 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:43.596329 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.597358 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.597364 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:43.597467 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:43.597493 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 21:10:43.597569 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 21:10:43.597639 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":93,"threads_started":1}
I20260806 21:10:43.597882 16225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:43.593243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:58771 (local address 127.27.225.195:45671)
0806 21:10:43.593435 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:43.593442 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:43.593457 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:10:43.593500 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:43.593503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:43.593549 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:43.593621 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:43.594268 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.594934 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.596456 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.597098 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.597212 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:43.597260 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:43.597622 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 21:10:43.597717 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260806 21:10:43.598150 16022 raft_consensus.cc:2454] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ae85a8e675e43c1835c72aac7313b61 in term 1.
I20260806 21:10:43.598274 15884 raft_consensus.cc:2454] T 1ef576fd7e254706acafaed0ab024d86 P 80cf00d2cc7e46bfbcfec726a5699d99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ae85a8e675e43c1835c72aac7313b61 in term 1.
I20260806 21:10:43.598532 16093 leader_election.cc:302] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ae85a8e675e43c1835c72aac7313b61, d37409d5e8b9498daa2a53ffa00aa085; no voters:
I20260806 21:10:43.598711 16233 raft_consensus.cc:2781] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:10:43.598948 16233 raft_consensus.cc:686] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61 [term 1 LEADER]: Becoming Leader. State: Replica: 9ae85a8e675e43c1835c72aac7313b61, State: Running, Role: LEADER
I20260806 21:10:43.599257 16160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ef576fd7e254706acafaed0ab024d86" candidate_uuid: "d37409d5e8b9498daa2a53ffa00aa085" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ae85a8e675e43c1835c72aac7313b61" is_pre_election: true
I20260806 21:10:43.599323 16233 consensus_queue.cc:237] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae85a8e675e43c1835c72aac7313b61" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 45671 } } peers { permanent_uuid: "d37409d5e8b9498daa2a53ffa00aa085" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 44843 } } peers { permanent_uuid: "80cf00d2cc7e46bfbcfec726a5699d99" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40381 } }
I20260806 21:10:43.600699 16224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:43.593882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:39835 (local address 127.27.225.193:40381)
0806 21:10:43.593998 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:43.594001 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:43.594491 (+ 490us) server_negotiation.cc:407] Connection header received
0806 21:10:43.594712 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:43.594719 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:43.594778 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:43.594845 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:43.595251 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.595817 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.599600 (+ 3783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.600202 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.600297 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:43.600333 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:43.600556 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 21:10:43.600607 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 21:10:43.600786 16239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:43.593531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:40381 (local address 127.27.225.194:39835)
0806 21:10:43.594393 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:43.594410 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:43.594573 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:43.594935 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:43.594940 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:43.594963 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:43.595106 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:43.595114 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.595975 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.595980 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:43.599406 (+ 3426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:43.599418 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.600326 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.600330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:43.600426 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:43.600454 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 21:10:43.600550 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 21:10:43.600633 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":793,"thread_start_us":85,"threads_started":1}
I20260806 21:10:43.601245 15884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ef576fd7e254706acafaed0ab024d86" candidate_uuid: "d37409d5e8b9498daa2a53ffa00aa085" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80cf00d2cc7e46bfbcfec726a5699d99" is_pre_election: true
I20260806 21:10:43.601428 15884 raft_consensus.cc:2379] T 1ef576fd7e254706acafaed0ab024d86 P 80cf00d2cc7e46bfbcfec726a5699d99 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d37409d5e8b9498daa2a53ffa00aa085 in current term 1: Already voted for candidate 9ae85a8e675e43c1835c72aac7313b61 in this term.
I20260806 21:10:43.601935 15956 leader_election.cc:302] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d37409d5e8b9498daa2a53ffa00aa085; no voters: 80cf00d2cc7e46bfbcfec726a5699d99, 9ae85a8e675e43c1835c72aac7313b61
I20260806 21:10:43.602226 16234 raft_consensus.cc:2726] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 21:10:43.603150 15734 catalog_manager.cc:5495] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ae85a8e675e43c1835c72aac7313b61 (127.27.225.195). New cstate: current_term: 1 leader_uuid: "9ae85a8e675e43c1835c72aac7313b61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ae85a8e675e43c1835c72aac7313b61" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 45671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d37409d5e8b9498daa2a53ffa00aa085" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 44843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80cf00d2cc7e46bfbcfec726a5699d99" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40381 } health_report { overall_health: UNKNOWN } } }
I20260806 21:10:43.657676 16225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:43.654096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55114 (local address 127.27.225.195:45671)
0806 21:10:43.654275 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:43.654279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:43.654294 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:10:43.654402 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:43.654404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:43.654460 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:43.654549 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:10:43.654798 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.655324 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.656186 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.656397 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.656475 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:43.656764 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:10:43.657219 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:10:43.657309 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:43.657422 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 21:10:43.657472 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 21:10:43.663923 15884 raft_consensus.cc:1261] T 1ef576fd7e254706acafaed0ab024d86 P 80cf00d2cc7e46bfbcfec726a5699d99 [term 1 FOLLOWER]: Refusing update from remote peer 9ae85a8e675e43c1835c72aac7313b61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:10:43.663964 16022 raft_consensus.cc:1261] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085 [term 1 FOLLOWER]: Refusing update from remote peer 9ae85a8e675e43c1835c72aac7313b61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:10:43.664621 16240 consensus_queue.cc:1035] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80cf00d2cc7e46bfbcfec726a5699d99" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 40381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:10:43.664837 16233 consensus_queue.cc:1035] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d37409d5e8b9498daa2a53ffa00aa085" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 44843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:10:43.672422 16245 mvcc.cc:205] Tried to move back new op lower bound from 7315663436439687168 to 7315663436186066944. Current Snapshot: MvccSnapshot[applied={T|T < 7315663436439687168}]
I20260806 21:10:43.673434 16244 mvcc.cc:205] Tried to move back new op lower bound from 7315663436439687168 to 7315663436186066944. Current Snapshot: MvccSnapshot[applied={T|T < 7315663436439687168}]
I20260806 21:10:43.680502 15735 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:10:43.680699 15735 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:10:43.683195 15735 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d086c3535bf843edbcb353e74d2eb545: Sending DeleteTablet for 3 replicas of tablet 1ef576fd7e254706acafaed0ab024d86
I20260806 21:10:43.684072 16139 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ef576fd7e254706acafaed0ab024d86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:10:43 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:55098
I20260806 21:10:43.684131 16002 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ef576fd7e254706acafaed0ab024d86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:10:43 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:39744
I20260806 21:10:43.684690 15863 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ef576fd7e254706acafaed0ab024d86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:10:43 UTC) from {username='kudu', principal='kudu/127.27.225.254@KRBTEST.COM'} at 127.0.0.1:43702
I20260806 21:10:43.684896 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 15799
I20260806 21:10:43.685899 16253 tablet_replica.cc:330] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085: stopping tablet replica
I20260806 21:10:43.686273 16253 raft_consensus.cc:2229] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:10:43.686647 16253 raft_consensus.cc:2258] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:10:43.687047 16252 tablet_replica.cc:330] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61: stopping tablet replica
I20260806 21:10:43.687345 16252 raft_consensus.cc:2229] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:10:43.687778 16252 raft_consensus.cc:2258] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:10:43.688512 16253 ts_tablet_manager.cc:1905] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:10:43.690840 16252 ts_tablet_manager.cc:1905] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:10:43.691463 16253 ts_tablet_manager.cc:1918] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:10:43.691553 16253 log.cc:1196] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-1/wal/wals/1ef576fd7e254706acafaed0ab024d86
I20260806 21:10:43.692225 16253 ts_tablet_manager.cc:1939] T 1ef576fd7e254706acafaed0ab024d86 P d37409d5e8b9498daa2a53ffa00aa085: Deleting consensus metadata
I20260806 21:10:43.693619 15720 catalog_manager.cc:4841] TS d37409d5e8b9498daa2a53ffa00aa085 (127.27.225.194:44843): tablet 1ef576fd7e254706acafaed0ab024d86 (table test-table [id=5272cc13d63e4e4dafb2794c64a0ac56]) successfully deleted
I20260806 21:10:43.694373 16252 ts_tablet_manager.cc:1918] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:10:43.694466 16252 log.cc:1196] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/1ef576fd7e254706acafaed0ab024d86
I20260806 21:10:43.694775 16252 ts_tablet_manager.cc:1939] T 1ef576fd7e254706acafaed0ab024d86 P 9ae85a8e675e43c1835c72aac7313b61: Deleting consensus metadata
I20260806 21:10:43.695828 15719 catalog_manager.cc:4841] TS 9ae85a8e675e43c1835c72aac7313b61 (127.27.225.195:45671): tablet 1ef576fd7e254706acafaed0ab024d86 (table test-table [id=5272cc13d63e4e4dafb2794c64a0ac56]) successfully deleted
W20260806 21:10:43.696254 15721 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.225.193:40381 (error 108)
I20260806 21:10:43.696529 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 15935
I20260806 21:10:43.699080 16222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:43.698759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:40381 (local address 127.0.0.1:43716)
0806 21:10:43.698944 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:43.699005 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.193:40381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260806 21:10:43.699307 15721 catalog_manager.cc:4584] TS 80cf00d2cc7e46bfbcfec726a5699d99 (127.27.225.193:40381): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1ef576fd7e254706acafaed0ab024d86: Network error: Client connection negotiation failed: client connection to 127.27.225.193:40381: connect: Connection refused (error 111)
I20260806 21:10:43.705178 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 16076
I20260806 21:10:43.712570 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 15703
2026-08-06T21:10:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2690 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T21:10:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:10:43Z Disabled control of system clock
I20260806 21:10:43.748626 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:46033
--webserver_interface=127.27.225.254
--webserver_port=0
--builtin_ntp_servers=127.27.225.212:42287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:46033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 21:10:43.865916 16258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:43.866226 16258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:43.866266 16258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:43.870520 16258 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:10:43.870597 16258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:43.870612 16258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:43.870630 16258 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:10:43.870644 16258 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:10:43.875929 16258 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:42287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:46033
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:46033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:43.877183 16258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:43.878752 16258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:43.887601 16266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:43.887606 16264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:43.887601 16263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:43.887858 16258 server_base.cc:1034] running on GCE node
I20260806 21:10:43.888556 16258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:43.889870 16258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:43.891091 16258 hybrid_clock.cc:648] HybridClock initialized: now 1786050643891070 us; error 34 us; skew 500 ppm
I20260806 21:10:43.893317 16258 webserver.cc:460] Webserver started at http://127.27.225.254:41739/ using document root <none> and password file <none>
I20260806 21:10:43.894083 16258 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:43.894192 16258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:43.894454 16258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:43.896793 16258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "96c4f31a34684cc9b6c83710e22fba69"
format_stamp: "Formatted at 2026-08-06 21:10:43 on dist-test-slave-69rg"
server_key: "1cdffbf069cc8c1e23b25de608b12000"
server_key_iv: "187644fa2de1b87b9159733070d61c0c"
server_key_version: "encryptionkey@0"
I20260806 21:10:43.897475 16258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "96c4f31a34684cc9b6c83710e22fba69"
format_stamp: "Formatted at 2026-08-06 21:10:43 on dist-test-slave-69rg"
server_key: "1cdffbf069cc8c1e23b25de608b12000"
server_key_iv: "187644fa2de1b87b9159733070d61c0c"
server_key_version: "encryptionkey@0"
I20260806 21:10:43.901664 16258 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260806 21:10:43.904417 16272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:43.905871 16258 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:10:43.906028 16258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "96c4f31a34684cc9b6c83710e22fba69"
format_stamp: "Formatted at 2026-08-06 21:10:43 on dist-test-slave-69rg"
server_key: "1cdffbf069cc8c1e23b25de608b12000"
server_key_iv: "187644fa2de1b87b9159733070d61c0c"
server_key_version: "encryptionkey@0"
I20260806 21:10:43.906149 16258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:43.924139 16258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:43.924902 16258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:43.925109 16258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:43.932368 16258 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:46033
I20260806 21:10:43.932379 16327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:46033 every 8 connection(s)
I20260806 21:10:43.933516 16258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:10:43.935943 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 16258
I20260806 21:10:43.936090 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:10:43.936424 28551 external_mini_cluster.cc:1428] Setting key 36f5d1da43e6a634099877cc229b0a2a
I20260806 21:10:43.936995 16328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:43.944478 16328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96c4f31a34684cc9b6c83710e22fba69: Bootstrap starting.
I20260806 21:10:43.947489 16328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96c4f31a34684cc9b6c83710e22fba69: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:43.947683 16331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:43.937944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41518 (local address 127.27.225.254:46033)
0806 21:10:43.938730 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:43.938750 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:43.938793 (+ 43us) server_negotiation.cc:407] Connection header received
0806 21:10:43.939831 (+ 1038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:43.939857 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:43.940380 (+ 523us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:43.940751 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:43.941233 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.942678 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.943581 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:43.944261 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:43.944437 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:43.945552 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:43.945559 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:43.945574 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:43.946130 (+ 556us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:43.946202 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:43.946230 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:43.946332 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:43.946474 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 21:10:43.946730 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":158,"threads_started":1}
I20260806 21:10:43.948342 16328 log.cc:824] T 00000000000000000000000000000000 P 96c4f31a34684cc9b6c83710e22fba69: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:43.950465 16328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96c4f31a34684cc9b6c83710e22fba69: No bootstrap required, opened a new log
I20260806 21:10:43.954286 16328 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96c4f31a34684cc9b6c83710e22fba69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96c4f31a34684cc9b6c83710e22fba69" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 46033 } }
I20260806 21:10:43.954530 16328 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96c4f31a34684cc9b6c83710e22fba69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:43.954599 16328 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96c4f31a34684cc9b6c83710e22fba69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96c4f31a34684cc9b6c83710e22fba69, State: Initialized, Role: FOLLOWER
I20260806 21:10:43.955066 16328 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96c4f31a34684cc9b6c83710e22fba69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96c4f31a34684cc9b6c83710e22fba69" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 46033 } }
I20260806 21:10:43.955194 16328 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96c4f31a34684cc9b6c83710e22fba69 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:10:43.955281 16328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96c4f31a34684cc9b6c83710e22fba69 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:10:43.955433 16328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96c4f31a34684cc9b6c83710e22fba69 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:43.956473 16328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96c4f31a34684cc9b6c83710e22fba69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96c4f31a34684cc9b6c83710e22fba69" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 46033 } }
I20260806 21:10:43.956835 16328 leader_election.cc:302] T 00000000000000000000000000000000 P 96c4f31a34684cc9b6c83710e22fba69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96c4f31a34684cc9b6c83710e22fba69; no voters:
I20260806 21:10:43.957216 16328 leader_election.cc:288] T 00000000000000000000000000000000 P 96c4f31a34684cc9b6c83710e22fba69 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:10:43.957417 16333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96c4f31a34684cc9b6c83710e22fba69 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:10:43.957697 16333 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96c4f31a34684cc9b6c83710e22fba69 [term 1 LEADER]: Becoming Leader. State: Replica: 96c4f31a34684cc9b6c83710e22fba69, State: Running, Role: LEADER
I20260806 21:10:43.958043 16333 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96c4f31a34684cc9b6c83710e22fba69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96c4f31a34684cc9b6c83710e22fba69" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 46033 } }
I20260806 21:10:43.958636 16328 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96c4f31a34684cc9b6c83710e22fba69 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:10:43.960057 16335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96c4f31a34684cc9b6c83710e22fba69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96c4f31a34684cc9b6c83710e22fba69. Latest consensus state: current_term: 1 leader_uuid: "96c4f31a34684cc9b6c83710e22fba69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96c4f31a34684cc9b6c83710e22fba69" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 46033 } } }
I20260806 21:10:43.960207 16335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96c4f31a34684cc9b6c83710e22fba69 [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:43.960078 16334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96c4f31a34684cc9b6c83710e22fba69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96c4f31a34684cc9b6c83710e22fba69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96c4f31a34684cc9b6c83710e22fba69" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 46033 } } }
I20260806 21:10:43.960358 16334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96c4f31a34684cc9b6c83710e22fba69 [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:43.960675 16342 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:10:43.964146 16342 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:10:43.970382 16342 catalog_manager.cc:1347] Generated new cluster ID: b05a73767f594f7e90a52406ff8f2819
I20260806 21:10:43.970499 16342 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:10:43.980911 16342 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:10:43.982275 16342 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:10:43.988061 16342 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96c4f31a34684cc9b6c83710e22fba69: Generated new TSK 0
I20260806 21:10:43.988950 16342 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:10:43.999990 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.193:0
--local_ip_for_outbound_sockets=127.27.225.193
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:46033
--builtin_ntp_servers=127.27.225.212:42287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 21:10:44.122259 16352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:44.122601 16352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:44.122677 16352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:44.127009 16352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:44.127115 16352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:44.127219 16352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.193
I20260806 21:10:44.132149 16352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:42287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.225.193
--webserver_port=0
--tserver_master_addrs=127.27.225.254:46033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.193
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:44.133355 16352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:44.134956 16352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:44.145051 16360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:44.145061 16357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:44.145289 16352 server_base.cc:1034] running on GCE node
W20260806 21:10:44.145468 16358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:44.145989 16352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:44.146657 16352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:44.147847 16352 hybrid_clock.cc:648] HybridClock initialized: now 1786050644147833 us; error 30 us; skew 500 ppm
I20260806 21:10:44.149755 16352 webserver.cc:460] Webserver started at http://127.27.225.193:38175/ using document root <none> and password file <none>
I20260806 21:10:44.150386 16352 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:44.150486 16352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:44.150724 16352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:44.152829 16352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-0/data/instance:
uuid: "bfb5e9d9a6264c17aa3b7a11f8914495"
format_stamp: "Formatted at 2026-08-06 21:10:44 on dist-test-slave-69rg"
server_key: "e30127a3e889599b86fb08a1c568600e"
server_key_iv: "a2e48c8f4bbe41b6edd8d65003d0d28a"
server_key_version: "encryptionkey@0"
I20260806 21:10:44.153419 16352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance:
uuid: "bfb5e9d9a6264c17aa3b7a11f8914495"
format_stamp: "Formatted at 2026-08-06 21:10:44 on dist-test-slave-69rg"
server_key: "e30127a3e889599b86fb08a1c568600e"
server_key_iv: "a2e48c8f4bbe41b6edd8d65003d0d28a"
server_key_version: "encryptionkey@0"
I20260806 21:10:44.157330 16352 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:10:44.160032 16366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:44.161341 16352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:10:44.161469 16352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-0/wal
uuid: "bfb5e9d9a6264c17aa3b7a11f8914495"
format_stamp: "Formatted at 2026-08-06 21:10:44 on dist-test-slave-69rg"
server_key: "e30127a3e889599b86fb08a1c568600e"
server_key_iv: "a2e48c8f4bbe41b6edd8d65003d0d28a"
server_key_version: "encryptionkey@0"
I20260806 21:10:44.161592 16352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:44.177316 16352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:44.178145 16352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:44.178351 16352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:44.179013 16352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:10:44.179991 16352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:10:44.180063 16352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:44.180140 16352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:10:44.180184 16352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:44.189620 16352 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.193:46445
I20260806 21:10:44.189647 16481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.193:46445 every 8 connection(s)
I20260806 21:10:44.190656 16352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-0/data/info.pb
I20260806 21:10:44.199195 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 16352
I20260806 21:10:44.199324 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-0/wal/instance
I20260806 21:10:44.199681 28551 external_mini_cluster.cc:1428] Setting key c92b0d89c2a373b1acd1228bef424a24
I20260806 21:10:44.200510 16331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:44.192599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.193:42027 (local address 127.27.225.254:46033)
0806 21:10:44.192745 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:44.192749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:44.193705 (+ 956us) server_negotiation.cc:407] Connection header received
0806 21:10:44.195216 (+ 1511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:44.195220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:44.195282 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:44.195395 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:44.196356 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.196903 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.198165 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.198408 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.198523 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:44.199969 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:44.199972 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:44.199975 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:44.199991 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:44.200011 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:44.200019 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:44.200093 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:44.200294 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 21:10:44.200392 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 21:10:44.201537 16484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:44.192916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:46033 (local address 127.27.225.193:42027)
0806 21:10:44.193557 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:44.193597 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:44.194991 (+ 1394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:44.195583 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:44.195591 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:44.195678 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:10:44.196192 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:44.196205 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.197038 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.197042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:44.198025 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:44.198036 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.198582 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.198586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:44.198752 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:44.199581 (+ 829us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:10:44.199583 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:10:44.199639 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:10:44.199727 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:10:44.200125 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:10:44.200138 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:10:44.200148 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 21:10:44.200254 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 21:10:44.200555 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":102,"threads_started":1}
I20260806 21:10:44.202164 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.194:0
--local_ip_for_outbound_sockets=127.27.225.194
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:46033
--builtin_ntp_servers=127.27.225.212:42287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 21:10:44.202907 16482 heartbeater.cc:347] Connected to a master server at 127.27.225.254:46033
I20260806 21:10:44.203167 16482 heartbeater.cc:464] Registering TS with master...
I20260806 21:10:44.203814 16482 heartbeater.cc:511] Master 127.27.225.254:46033 requested a full tablet report, sending...
I20260806 21:10:44.205675 16290 ts_manager.cc:194] Registered new tserver with Master: bfb5e9d9a6264c17aa3b7a11f8914495 (127.27.225.193:46445)
I20260806 21:10:44.207434 16290 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.225.193:42027
W20260806 21:10:44.318163 16485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:44.318452 16485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:44.318492 16485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:44.322805 16485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:44.322870 16485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:44.322927 16485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.194
I20260806 21:10:44.327862 16485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:42287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.225.194
--webserver_port=0
--tserver_master_addrs=127.27.225.254:46033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.194
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:44.329077 16485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:44.330602 16485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:44.340219 16491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:44.340211 16493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:44.340211 16490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:44.340742 16485 server_base.cc:1034] running on GCE node
I20260806 21:10:44.341240 16485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:44.341933 16485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:44.343118 16485 hybrid_clock.cc:648] HybridClock initialized: now 1786050644343106 us; error 28 us; skew 500 ppm
I20260806 21:10:44.344939 16485 webserver.cc:460] Webserver started at http://127.27.225.194:41533/ using document root <none> and password file <none>
I20260806 21:10:44.345551 16485 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:44.345614 16485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:44.345929 16485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:44.348006 16485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-1/data/instance:
uuid: "49b34aebc28742248494e6334751c399"
format_stamp: "Formatted at 2026-08-06 21:10:44 on dist-test-slave-69rg"
server_key: "6a130a705c15f201f3feabd845f56bb0"
server_key_iv: "a1ddc2a68f1c81c3fd5fadc3fcd2cc2c"
server_key_version: "encryptionkey@0"
I20260806 21:10:44.348587 16485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance:
uuid: "49b34aebc28742248494e6334751c399"
format_stamp: "Formatted at 2026-08-06 21:10:44 on dist-test-slave-69rg"
server_key: "6a130a705c15f201f3feabd845f56bb0"
server_key_iv: "a1ddc2a68f1c81c3fd5fadc3fcd2cc2c"
server_key_version: "encryptionkey@0"
I20260806 21:10:44.352430 16485 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 21:10:44.355135 16499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:44.356478 16485 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 21:10:44.356621 16485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-1/wal
uuid: "49b34aebc28742248494e6334751c399"
format_stamp: "Formatted at 2026-08-06 21:10:44 on dist-test-slave-69rg"
server_key: "6a130a705c15f201f3feabd845f56bb0"
server_key_iv: "a1ddc2a68f1c81c3fd5fadc3fcd2cc2c"
server_key_version: "encryptionkey@0"
I20260806 21:10:44.356739 16485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:44.377578 16485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:44.378386 16485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:44.378608 16485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:44.379240 16485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:10:44.380241 16485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:10:44.380316 16485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:44.380394 16485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:10:44.380437 16485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:44.389694 16485 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.194:37583
I20260806 21:10:44.389717 16613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.194:37583 every 8 connection(s)
I20260806 21:10:44.390695 16485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-1/data/info.pb
I20260806 21:10:44.400314 16331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:44.392714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:35131 (local address 127.27.225.254:46033)
0806 21:10:44.392872 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:44.392877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:44.393937 (+ 1060us) server_negotiation.cc:407] Connection header received
0806 21:10:44.395440 (+ 1503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:44.395443 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:44.395511 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:44.395627 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:44.396566 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.397127 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.398352 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.398582 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.398696 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:44.399791 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:44.399795 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:44.399797 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:44.399825 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:44.399845 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:44.399853 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:44.399933 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:44.400094 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 21:10:44.400159 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 21:10:44.400856 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 16485
I20260806 21:10:44.401010 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-1/wal/instance
I20260806 21:10:44.401362 16616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:44.393099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:46033 (local address 127.27.225.194:35131)
0806 21:10:44.393752 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:44.393794 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:44.395196 (+ 1402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:44.395814 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:44.395821 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:44.395908 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:10:44.396408 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:44.396421 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.397272 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.397275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:44.398200 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:44.398211 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.398743 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.398747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:44.398912 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:44.399524 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:10:44.399525 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:10:44.399567 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:10:44.399631 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:10:44.399934 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:10:44.399945 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:10:44.399954 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 21:10:44.400093 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 21:10:44.400387 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":112,"threads_started":1}
I20260806 21:10:44.401381 28551 external_mini_cluster.cc:1428] Setting key 4039205a763fd82bd9d481f26fdf419a
I20260806 21:10:44.402586 16614 heartbeater.cc:347] Connected to a master server at 127.27.225.254:46033
I20260806 21:10:44.402854 16614 heartbeater.cc:464] Registering TS with master...
I20260806 21:10:44.403395 16614 heartbeater.cc:511] Master 127.27.225.254:46033 requested a full tablet report, sending...
I20260806 21:10:44.403455 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.225.195:0
--local_ip_for_outbound_sockets=127.27.225.195
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:46033
--builtin_ntp_servers=127.27.225.212:42287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 21:10:44.404628 16290 ts_manager.cc:194] Registered new tserver with Master: 49b34aebc28742248494e6334751c399 (127.27.225.194:37583)
I20260806 21:10:44.405479 16290 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.225.194:35131
W20260806 21:10:44.520852 16617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:44.521201 16617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:44.521281 16617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:44.525635 16617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:44.525763 16617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:44.525892 16617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.225.195
I20260806 21:10:44.531282 16617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:42287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.225.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.225.195
--webserver_port=0
--tserver_master_addrs=127.27.225.254:46033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.225.195
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:44.532506 16617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:44.534018 16617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:44.544121 16625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:44.544227 16617 server_base.cc:1034] running on GCE node
W20260806 21:10:44.544123 16623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:44.544122 16622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:44.544849 16617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:44.545485 16617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:44.546730 16617 hybrid_clock.cc:648] HybridClock initialized: now 1786050644546707 us; error 43 us; skew 500 ppm
I20260806 21:10:44.548991 16617 webserver.cc:460] Webserver started at http://127.27.225.195:34485/ using document root <none> and password file <none>
I20260806 21:10:44.549587 16617 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:44.549691 16617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:44.549974 16617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:44.552237 16617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-2/data/instance:
uuid: "2f9fc6cc4ee84efa9ac409aa052eeb1b"
format_stamp: "Formatted at 2026-08-06 21:10:44 on dist-test-slave-69rg"
server_key: "112103e8116e84a4bed865153ca1fc07"
server_key_iv: "d7088661b7caaf400d8852f368924f39"
server_key_version: "encryptionkey@0"
I20260806 21:10:44.552852 16617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance:
uuid: "2f9fc6cc4ee84efa9ac409aa052eeb1b"
format_stamp: "Formatted at 2026-08-06 21:10:44 on dist-test-slave-69rg"
server_key: "112103e8116e84a4bed865153ca1fc07"
server_key_iv: "d7088661b7caaf400d8852f368924f39"
server_key_version: "encryptionkey@0"
I20260806 21:10:44.556734 16617 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 21:10:44.559520 16631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:44.561116 16617 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 21:10:44.561223 16617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-2/wal
uuid: "2f9fc6cc4ee84efa9ac409aa052eeb1b"
format_stamp: "Formatted at 2026-08-06 21:10:44 on dist-test-slave-69rg"
server_key: "112103e8116e84a4bed865153ca1fc07"
server_key_iv: "d7088661b7caaf400d8852f368924f39"
server_key_version: "encryptionkey@0"
I20260806 21:10:44.561374 16617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:44.581744 16617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:44.582556 16617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:44.582772 16617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:44.583424 16617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:10:44.584421 16617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:10:44.584468 16617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:44.584532 16617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:10:44.584566 16617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:44.593586 16617 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.195:34397
I20260806 21:10:44.593609 16748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.195:34397 every 8 connection(s)
I20260806 21:10:44.594539 16617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-2/data/info.pb
I20260806 21:10:44.602161 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 16617
I20260806 21:10:44.602319 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-2/wal/instance
I20260806 21:10:44.602636 28551 external_mini_cluster.cc:1428] Setting key 3b0b29c23b44ae8e94f24f3f168bd62d
I20260806 21:10:44.604215 16331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:44.596481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.195:49447 (local address 127.27.225.254:46033)
0806 21:10:44.596622 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:44.596626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:44.597673 (+ 1047us) server_negotiation.cc:407] Connection header received
0806 21:10:44.599170 (+ 1497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:44.599173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:44.599236 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:44.599337 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:44.600237 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.600775 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.601991 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.602217 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.602334 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:44.603707 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:44.603711 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:44.603713 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:44.603726 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:44.603746 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:44.603754 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:44.603836 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:44.604006 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 21:10:44.604076 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 21:10:44.605229 16751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:44.596865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:46033 (local address 127.27.225.195:49447)
0806 21:10:44.597520 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:44.597561 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:44.598949 (+ 1388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:44.599516 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:44.599523 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:44.599604 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:10:44.600081 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:44.600094 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.600918 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.600924 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:44.601799 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:44.601810 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.602385 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.602389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:44.602559 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:44.603359 (+ 800us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:10:44.603366 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:10:44.603423 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:10:44.603512 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:10:44.603838 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:10:44.603850 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:10:44.603859 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 21:10:44.604005 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 21:10:44.604321 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":131,"threads_started":1}
I20260806 21:10:44.606340 16749 heartbeater.cc:347] Connected to a master server at 127.27.225.254:46033
I20260806 21:10:44.606581 16749 heartbeater.cc:464] Registering TS with master...
I20260806 21:10:44.607128 16749 heartbeater.cc:511] Master 127.27.225.254:46033 requested a full tablet report, sending...
I20260806 21:10:44.608088 16290 ts_manager.cc:194] Registered new tserver with Master: 2f9fc6cc4ee84efa9ac409aa052eeb1b (127.27.225.195:34397)
I20260806 21:10:44.608664 16290 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.225.195:49447
I20260806 21:10:44.616824 28551 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:10:44.622502 16331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:44.618840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41522 (local address 127.27.225.254:46033)
0806 21:10:44.619006 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:44.619010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:44.619141 (+ 131us) server_negotiation.cc:407] Connection header received
0806 21:10:44.619244 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:44.619246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:44.619289 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:44.619355 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:44.619617 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.620188 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.620987 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.621221 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.621297 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:44.622080 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:44.622083 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:44.622085 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:44.622097 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:44.622116 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:44.622123 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:44.622211 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:44.622275 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 21:10:44.622324 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 21:10:44.625275 16290 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41522:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:10:44.628341 16290 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:10:44.643466 16760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:44.636929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:34397 (local address 127.0.0.1:41614)
0806 21:10:44.637705 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:44.637943 (+ 238us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:44.638096 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:44.638966 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:44.638973 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:44.638995 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:44.639240 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:44.639248 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.640729 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.640732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:44.642180 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:44.642187 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.642994 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.642997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:44.643066 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:44.643083 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 21:10:44.643141 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 21:10:44.643296 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":181,"spinlock_wait_cycles":7040,"thread_start_us":95,"threads_started":1}
I20260806 21:10:44.643638 16764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:44.637855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32992 (local address 127.27.225.194:37583)
0806 21:10:44.638467 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:44.638479 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:44.638507 (+ 28us) server_negotiation.cc:407] Connection header received
0806 21:10:44.638587 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:44.638593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:44.638793 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:44.638950 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:44.639405 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.640153 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.642310 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.643053 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.643186 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:44.643341 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:44.643390 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 21:10:44.643478 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":134,"threads_started":1}
I20260806 21:10:44.643532 16765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:44.638294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43314 (local address 127.27.225.193:46445)
0806 21:10:44.638727 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:44.638736 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:44.638753 (+ 17us) server_negotiation.cc:407] Connection header received
0806 21:10:44.638812 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:44.638815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:44.638993 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:44.639149 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:44.639482 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.640602 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.642020 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.642749 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.642876 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:44.643058 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:44.643273 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 21:10:44.643372 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":123,"threads_started":1}
I20260806 21:10:44.643637 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:44.637142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.194:37583 (local address 127.0.0.1:32992)
0806 21:10:44.637875 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:44.637938 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:44.638095 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:44.639115 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:44.639117 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:44.639132 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:44.639308 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:44.639316 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.640286 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.640290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:44.642164 (+ 1874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:44.642178 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.643172 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.643176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:44.643270 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:44.643291 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 21:10:44.643355 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 21:10:44.643425 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"mutex_wait_us":188,"spinlock_wait_cycles":4352,"thread_start_us":131,"threads_started":1}
I20260806 21:10:44.644433 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:44.637148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41614 (local address 127.27.225.195:34397)
0806 21:10:44.638498 (+ 1350us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:44.638509 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:44.638529 (+ 20us) server_negotiation.cc:407] Connection header received
0806 21:10:44.638593 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:44.638596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:44.638793 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:44.638950 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:44.639371 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.640595 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.642323 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.642867 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.643881 (+ 1014us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:44.644075 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:44.644130 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 21:10:44.644258 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1257,"thread_start_us":135,"threads_started":1}
I20260806 21:10:44.644761 16763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:44.637611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:46445 (local address 127.0.0.1:43314)
0806 21:10:44.638472 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:44.638486 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:44.638570 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:44.639216 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:44.639219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:44.639234 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:44.639351 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:44.639358 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.640729 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.640732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:44.641895 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:44.641906 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.642888 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.642895 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:44.643016 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:44.643058 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 21:10:44.643163 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 21:10:44.644638 (+ 1475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":798,"mutex_wait_us":1422,"thread_start_us":105,"threads_started":1}
I20260806 21:10:44.647081 16682 tablet_service.cc:1467] Processing CreateTablet for tablet 52eff4fe777c41d2a7872d52b8d9cddc (DEFAULT_TABLE table=test-table [id=179b874c78a74bb78d4695e403b87982]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:10:44.647109 16548 tablet_service.cc:1467] Processing CreateTablet for tablet 52eff4fe777c41d2a7872d52b8d9cddc (DEFAULT_TABLE table=test-table [id=179b874c78a74bb78d4695e403b87982]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:10:44.647119 16415 tablet_service.cc:1467] Processing CreateTablet for tablet 52eff4fe777c41d2a7872d52b8d9cddc (DEFAULT_TABLE table=test-table [id=179b874c78a74bb78d4695e403b87982]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:10:44.648090 16682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52eff4fe777c41d2a7872d52b8d9cddc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:44.648113 16548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52eff4fe777c41d2a7872d52b8d9cddc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:44.648115 16415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52eff4fe777c41d2a7872d52b8d9cddc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:44.654116 16766 tablet_bootstrap.cc:492] T 52eff4fe777c41d2a7872d52b8d9cddc P 2f9fc6cc4ee84efa9ac409aa052eeb1b: Bootstrap starting.
I20260806 21:10:44.655579 16768 tablet_bootstrap.cc:492] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399: Bootstrap starting.
I20260806 21:10:44.656879 16766 tablet_bootstrap.cc:654] T 52eff4fe777c41d2a7872d52b8d9cddc P 2f9fc6cc4ee84efa9ac409aa052eeb1b: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:44.657219 16768 tablet_bootstrap.cc:654] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:44.657631 16766 log.cc:824] T 52eff4fe777c41d2a7872d52b8d9cddc P 2f9fc6cc4ee84efa9ac409aa052eeb1b: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:44.657975 16768 log.cc:824] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:44.658279 16767 tablet_bootstrap.cc:492] T 52eff4fe777c41d2a7872d52b8d9cddc P bfb5e9d9a6264c17aa3b7a11f8914495: Bootstrap starting.
I20260806 21:10:44.659725 16766 tablet_bootstrap.cc:492] T 52eff4fe777c41d2a7872d52b8d9cddc P 2f9fc6cc4ee84efa9ac409aa052eeb1b: No bootstrap required, opened a new log
I20260806 21:10:44.659940 16766 ts_tablet_manager.cc:1397] T 52eff4fe777c41d2a7872d52b8d9cddc P 2f9fc6cc4ee84efa9ac409aa052eeb1b: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260806 21:10:44.659962 16768 tablet_bootstrap.cc:492] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399: No bootstrap required, opened a new log
I20260806 21:10:44.660123 16768 ts_tablet_manager.cc:1397] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260806 21:10:44.660488 16767 tablet_bootstrap.cc:654] T 52eff4fe777c41d2a7872d52b8d9cddc P bfb5e9d9a6264c17aa3b7a11f8914495: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:44.661273 16767 log.cc:824] T 52eff4fe777c41d2a7872d52b8d9cddc P bfb5e9d9a6264c17aa3b7a11f8914495: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:44.663282 16767 tablet_bootstrap.cc:492] T 52eff4fe777c41d2a7872d52b8d9cddc P bfb5e9d9a6264c17aa3b7a11f8914495: No bootstrap required, opened a new log
I20260806 21:10:44.663475 16767 ts_tablet_manager.cc:1397] T 52eff4fe777c41d2a7872d52b8d9cddc P bfb5e9d9a6264c17aa3b7a11f8914495: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 21:10:44.663923 16766 raft_consensus.cc:348] T 52eff4fe777c41d2a7872d52b8d9cddc P 2f9fc6cc4ee84efa9ac409aa052eeb1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9fc6cc4ee84efa9ac409aa052eeb1b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34397 } } peers { permanent_uuid: "49b34aebc28742248494e6334751c399" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37583 } } peers { permanent_uuid: "bfb5e9d9a6264c17aa3b7a11f8914495" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 46445 } }
I20260806 21:10:44.664170 16766 raft_consensus.cc:374] T 52eff4fe777c41d2a7872d52b8d9cddc P 2f9fc6cc4ee84efa9ac409aa052eeb1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:44.664230 16766 raft_consensus.cc:729] T 52eff4fe777c41d2a7872d52b8d9cddc P 2f9fc6cc4ee84efa9ac409aa052eeb1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9fc6cc4ee84efa9ac409aa052eeb1b, State: Initialized, Role: FOLLOWER
I20260806 21:10:44.664394 16768 raft_consensus.cc:348] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9fc6cc4ee84efa9ac409aa052eeb1b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34397 } } peers { permanent_uuid: "49b34aebc28742248494e6334751c399" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37583 } } peers { permanent_uuid: "bfb5e9d9a6264c17aa3b7a11f8914495" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 46445 } }
I20260806 21:10:44.664644 16768 raft_consensus.cc:374] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:44.664701 16768 raft_consensus.cc:729] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49b34aebc28742248494e6334751c399, State: Initialized, Role: FOLLOWER
I20260806 21:10:44.664753 16766 consensus_queue.cc:260] T 52eff4fe777c41d2a7872d52b8d9cddc P 2f9fc6cc4ee84efa9ac409aa052eeb1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9fc6cc4ee84efa9ac409aa052eeb1b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34397 } } peers { permanent_uuid: "49b34aebc28742248494e6334751c399" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37583 } } peers { permanent_uuid: "bfb5e9d9a6264c17aa3b7a11f8914495" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 46445 } }
I20260806 21:10:44.665095 16768 consensus_queue.cc:260] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9fc6cc4ee84efa9ac409aa052eeb1b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34397 } } peers { permanent_uuid: "49b34aebc28742248494e6334751c399" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37583 } } peers { permanent_uuid: "bfb5e9d9a6264c17aa3b7a11f8914495" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 46445 } }
I20260806 21:10:44.665630 16749 heartbeater.cc:503] Master 127.27.225.254:46033 was elected leader, sending a full tablet report...
I20260806 21:10:44.665926 16766 ts_tablet_manager.cc:1428] T 52eff4fe777c41d2a7872d52b8d9cddc P 2f9fc6cc4ee84efa9ac409aa052eeb1b: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 21:10:44.666030 16768 ts_tablet_manager.cc:1428] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260806 21:10:44.666368 16614 heartbeater.cc:503] Master 127.27.225.254:46033 was elected leader, sending a full tablet report...
I20260806 21:10:44.667761 16767 raft_consensus.cc:348] T 52eff4fe777c41d2a7872d52b8d9cddc P bfb5e9d9a6264c17aa3b7a11f8914495 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9fc6cc4ee84efa9ac409aa052eeb1b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34397 } } peers { permanent_uuid: "49b34aebc28742248494e6334751c399" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37583 } } peers { permanent_uuid: "bfb5e9d9a6264c17aa3b7a11f8914495" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 46445 } }
I20260806 21:10:44.667937 16767 raft_consensus.cc:374] T 52eff4fe777c41d2a7872d52b8d9cddc P bfb5e9d9a6264c17aa3b7a11f8914495 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:44.667968 16767 raft_consensus.cc:729] T 52eff4fe777c41d2a7872d52b8d9cddc P bfb5e9d9a6264c17aa3b7a11f8914495 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfb5e9d9a6264c17aa3b7a11f8914495, State: Initialized, Role: FOLLOWER
I20260806 21:10:44.668257 16767 consensus_queue.cc:260] T 52eff4fe777c41d2a7872d52b8d9cddc P bfb5e9d9a6264c17aa3b7a11f8914495 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9fc6cc4ee84efa9ac409aa052eeb1b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34397 } } peers { permanent_uuid: "49b34aebc28742248494e6334751c399" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37583 } } peers { permanent_uuid: "bfb5e9d9a6264c17aa3b7a11f8914495" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 46445 } }
I20260806 21:10:44.668823 16482 heartbeater.cc:503] Master 127.27.225.254:46033 was elected leader, sending a full tablet report...
I20260806 21:10:44.670207 16767 ts_tablet_manager.cc:1428] T 52eff4fe777c41d2a7872d52b8d9cddc P bfb5e9d9a6264c17aa3b7a11f8914495: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 21:10:44.679296 16774 raft_consensus.cc:482] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:10:44.679497 16774 raft_consensus.cc:504] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9fc6cc4ee84efa9ac409aa052eeb1b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34397 } } peers { permanent_uuid: "49b34aebc28742248494e6334751c399" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37583 } } peers { permanent_uuid: "bfb5e9d9a6264c17aa3b7a11f8914495" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 46445 } }
I20260806 21:10:44.680691 16774 leader_election.cc:288] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9fc6cc4ee84efa9ac409aa052eeb1b (127.27.225.195:34397), bfb5e9d9a6264c17aa3b7a11f8914495 (127.27.225.193:46445)
I20260806 21:10:44.684568 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:44.681050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:42045 (local address 127.27.225.195:34397)
0806 21:10:44.681190 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:44.681194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:44.681207 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:10:44.681262 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:44.681265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:44.681311 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:44.681396 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:44.681744 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.682396 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.683517 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.683939 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.684029 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:44.684066 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:44.684366 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 21:10:44.684442 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 21:10:44.684563 16616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:44.680835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.195:34397 (local address 127.27.225.194:42045)
0806 21:10:44.680987 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:44.681004 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:44.681094 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:44.681443 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:44.681446 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:44.681471 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:44.681617 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:44.681624 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.682509 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.682512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:44.683394 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:44.683403 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.684055 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.684061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:44.684216 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:44.684256 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 21:10:44.684325 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 21:10:44.684412 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":20}
I20260806 21:10:44.684600 16765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:44.681057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.225.194:48069 (local address 127.27.225.193:46445)
0806 21:10:44.681182 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:44.681187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:44.681444 (+ 257us) server_negotiation.cc:407] Connection header received
0806 21:10:44.681559 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:44.681562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:44.681622 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:44.681703 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:10:44.682016 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.682560 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.683628 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.684034 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.684160 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:44.684211 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:44.684419 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 21:10:44.684493 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 21:10:44.684585 16776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:44.680939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:46445 (local address 127.27.225.194:48069)
0806 21:10:44.681347 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:44.681362 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:44.681449 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:44.681741 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:10:44.681744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:10:44.681761 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:10:44.681897 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:44.681904 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.682689 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.682691 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:44.683502 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:10:44.683512 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.684156 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.684159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:10:44.684277 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:44.684293 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 21:10:44.684388 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 21:10:44.684447 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":51,"thread_start_us":164,"threads_started":1}
I20260806 21:10:44.685328 16702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52eff4fe777c41d2a7872d52b8d9cddc" candidate_uuid: "49b34aebc28742248494e6334751c399" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9fc6cc4ee84efa9ac409aa052eeb1b" is_pre_election: true
I20260806 21:10:44.685635 16702 raft_consensus.cc:2454] T 52eff4fe777c41d2a7872d52b8d9cddc P 2f9fc6cc4ee84efa9ac409aa052eeb1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49b34aebc28742248494e6334751c399 in term 0.
I20260806 21:10:44.686201 16500 leader_election.cc:302] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9fc6cc4ee84efa9ac409aa052eeb1b, 49b34aebc28742248494e6334751c399; no voters:
I20260806 21:10:44.686293 16435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52eff4fe777c41d2a7872d52b8d9cddc" candidate_uuid: "49b34aebc28742248494e6334751c399" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfb5e9d9a6264c17aa3b7a11f8914495" is_pre_election: true
I20260806 21:10:44.686482 16774 raft_consensus.cc:2781] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:10:44.686585 16774 raft_consensus.cc:482] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:10:44.686587 16435 raft_consensus.cc:2454] T 52eff4fe777c41d2a7872d52b8d9cddc P bfb5e9d9a6264c17aa3b7a11f8914495 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49b34aebc28742248494e6334751c399 in term 0.
I20260806 21:10:44.686645 16774 raft_consensus.cc:3037] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:44.687897 16774 raft_consensus.cc:504] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9fc6cc4ee84efa9ac409aa052eeb1b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34397 } } peers { permanent_uuid: "49b34aebc28742248494e6334751c399" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37583 } } peers { permanent_uuid: "bfb5e9d9a6264c17aa3b7a11f8914495" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 46445 } }
I20260806 21:10:44.688314 16774 leader_election.cc:288] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399 [CANDIDATE]: Term 1 election: Requested vote from peers 2f9fc6cc4ee84efa9ac409aa052eeb1b (127.27.225.195:34397), bfb5e9d9a6264c17aa3b7a11f8914495 (127.27.225.193:46445)
I20260806 21:10:44.688632 16702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52eff4fe777c41d2a7872d52b8d9cddc" candidate_uuid: "49b34aebc28742248494e6334751c399" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9fc6cc4ee84efa9ac409aa052eeb1b"
I20260806 21:10:44.688671 16435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52eff4fe777c41d2a7872d52b8d9cddc" candidate_uuid: "49b34aebc28742248494e6334751c399" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfb5e9d9a6264c17aa3b7a11f8914495"
I20260806 21:10:44.688773 16702 raft_consensus.cc:3037] T 52eff4fe777c41d2a7872d52b8d9cddc P 2f9fc6cc4ee84efa9ac409aa052eeb1b [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:44.688804 16435 raft_consensus.cc:3037] T 52eff4fe777c41d2a7872d52b8d9cddc P bfb5e9d9a6264c17aa3b7a11f8914495 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:44.689774 16702 raft_consensus.cc:2454] T 52eff4fe777c41d2a7872d52b8d9cddc P 2f9fc6cc4ee84efa9ac409aa052eeb1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49b34aebc28742248494e6334751c399 in term 1.
I20260806 21:10:44.689903 16435 raft_consensus.cc:2454] T 52eff4fe777c41d2a7872d52b8d9cddc P bfb5e9d9a6264c17aa3b7a11f8914495 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49b34aebc28742248494e6334751c399 in term 1.
I20260806 21:10:44.690107 16500 leader_election.cc:302] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9fc6cc4ee84efa9ac409aa052eeb1b, 49b34aebc28742248494e6334751c399; no voters:
I20260806 21:10:44.690285 16774 raft_consensus.cc:2781] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:10:44.690485 16774 raft_consensus.cc:686] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399 [term 1 LEADER]: Becoming Leader. State: Replica: 49b34aebc28742248494e6334751c399, State: Running, Role: LEADER
I20260806 21:10:44.690819 16774 consensus_queue.cc:237] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9fc6cc4ee84efa9ac409aa052eeb1b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34397 } } peers { permanent_uuid: "49b34aebc28742248494e6334751c399" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37583 } } peers { permanent_uuid: "bfb5e9d9a6264c17aa3b7a11f8914495" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 46445 } }
W20260806 21:10:44.691856 16483 tablet.cc:2369] T 52eff4fe777c41d2a7872d52b8d9cddc P bfb5e9d9a6264c17aa3b7a11f8914495: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:10:44.694233 16290 catalog_manager.cc:5495] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49b34aebc28742248494e6334751c399 (127.27.225.194). New cstate: current_term: 1 leader_uuid: "49b34aebc28742248494e6334751c399" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9fc6cc4ee84efa9ac409aa052eeb1b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49b34aebc28742248494e6334751c399" member_type: VOTER last_known_addr { host: "127.27.225.194" port: 37583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfb5e9d9a6264c17aa3b7a11f8914495" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 46445 } health_report { overall_health: UNKNOWN } } }
I20260806 21:10:44.709331 16764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:44.706077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32994 (local address 127.27.225.194:37583)
0806 21:10:44.706231 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:44.706235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:44.706249 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:10:44.706339 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:44.706341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:44.706394 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:44.706483 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:10:44.706697 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.707187 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.707980 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:44.708198 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:44.708277 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:44.708537 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:10:44.708945 (+ 408us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:10:44.709034 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:44.709093 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 21:10:44.709140 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 21:10:44.715390 16435 raft_consensus.cc:1261] T 52eff4fe777c41d2a7872d52b8d9cddc P bfb5e9d9a6264c17aa3b7a11f8914495 [term 1 FOLLOWER]: Refusing update from remote peer 49b34aebc28742248494e6334751c399: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:10:44.715390 16702 raft_consensus.cc:1261] T 52eff4fe777c41d2a7872d52b8d9cddc P 2f9fc6cc4ee84efa9ac409aa052eeb1b [term 1 FOLLOWER]: Refusing update from remote peer 49b34aebc28742248494e6334751c399: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:10:44.716075 16774 consensus_queue.cc:1035] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9fc6cc4ee84efa9ac409aa052eeb1b" member_type: VOTER last_known_addr { host: "127.27.225.195" port: 34397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:10:44.716313 16777 consensus_queue.cc:1035] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb5e9d9a6264c17aa3b7a11f8914495" member_type: VOTER last_known_addr { host: "127.27.225.193" port: 46445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:10:44.724282 16780 mvcc.cc:205] Tried to move back new op lower bound from 7315663440746659840 to 7315663440656240640. Current Snapshot: MvccSnapshot[applied={T|T < 7315663440746659840}]
I20260806 21:10:44.726169 16782 mvcc.cc:205] Tried to move back new op lower bound from 7315663440746659840 to 7315663440656240640. Current Snapshot: MvccSnapshot[applied={T|T < 7315663440746659840}]
I20260806 21:10:44.732770 16289 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:10:44.732972 16289 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:10:44.735621 16289 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 96c4f31a34684cc9b6c83710e22fba69: Sending DeleteTablet for 3 replicas of tablet 52eff4fe777c41d2a7872d52b8d9cddc
I20260806 21:10:44.736510 16682 tablet_service.cc:1514] Processing DeleteTablet for tablet 52eff4fe777c41d2a7872d52b8d9cddc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:10:44 UTC) from {username='slave'} at 127.0.0.1:41614
I20260806 21:10:44.736569 16548 tablet_service.cc:1514] Processing DeleteTablet for tablet 52eff4fe777c41d2a7872d52b8d9cddc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:10:44 UTC) from {username='slave'} at 127.0.0.1:32992
I20260806 21:10:44.736759 16415 tablet_service.cc:1514] Processing DeleteTablet for tablet 52eff4fe777c41d2a7872d52b8d9cddc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:10:44 UTC) from {username='slave'} at 127.0.0.1:43314
I20260806 21:10:44.737130 16791 tablet_replica.cc:330] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399: stopping tablet replica
I20260806 21:10:44.737294 16792 tablet_replica.cc:330] T 52eff4fe777c41d2a7872d52b8d9cddc P bfb5e9d9a6264c17aa3b7a11f8914495: stopping tablet replica
I20260806 21:10:44.737596 16791 raft_consensus.cc:2229] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:10:44.737733 16792 raft_consensus.cc:2229] T 52eff4fe777c41d2a7872d52b8d9cddc P bfb5e9d9a6264c17aa3b7a11f8914495 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:10:44.737982 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 16352
I20260806 21:10:44.738128 16792 raft_consensus.cc:2258] T 52eff4fe777c41d2a7872d52b8d9cddc P bfb5e9d9a6264c17aa3b7a11f8914495 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:10:44.740329 16791 raft_consensus.cc:2258] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:10:44.741132 16790 tablet_replica.cc:330] T 52eff4fe777c41d2a7872d52b8d9cddc P 2f9fc6cc4ee84efa9ac409aa052eeb1b: stopping tablet replica
I20260806 21:10:44.741600 16790 raft_consensus.cc:2229] T 52eff4fe777c41d2a7872d52b8d9cddc P 2f9fc6cc4ee84efa9ac409aa052eeb1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:10:44.742095 16790 raft_consensus.cc:2258] T 52eff4fe777c41d2a7872d52b8d9cddc P 2f9fc6cc4ee84efa9ac409aa052eeb1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:10:44.742605 16791 ts_tablet_manager.cc:1905] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:10:44.747032 16791 ts_tablet_manager.cc:1918] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:10:44.747201 16791 log.cc:1196] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-1/wal/wals/52eff4fe777c41d2a7872d52b8d9cddc
I20260806 21:10:44.747584 16791 ts_tablet_manager.cc:1939] T 52eff4fe777c41d2a7872d52b8d9cddc P 49b34aebc28742248494e6334751c399: Deleting consensus metadata
I20260806 21:10:44.748075 16790 ts_tablet_manager.cc:1905] T 52eff4fe777c41d2a7872d52b8d9cddc P 2f9fc6cc4ee84efa9ac409aa052eeb1b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 21:10:44.748281 16274 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.225.193:46445 (error 108)
I20260806 21:10:44.748994 16274 catalog_manager.cc:4841] TS 49b34aebc28742248494e6334751c399 (127.27.225.194:37583): tablet 52eff4fe777c41d2a7872d52b8d9cddc (table test-table [id=179b874c78a74bb78d4695e403b87982]) successfully deleted
W20260806 21:10:44.749902 16763 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:10:44.749345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.193:46445 (local address 127.0.0.1:43316)
0806 21:10:44.749550 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 21:10:44.749566 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:10:44.749663 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:10:44.749767 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.193:46445: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":104}
W20260806 21:10:44.750092 16274 catalog_manager.cc:4584] TS bfb5e9d9a6264c17aa3b7a11f8914495 (127.27.225.193:46445): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 52eff4fe777c41d2a7872d52b8d9cddc: Network error: Client connection negotiation failed: client connection to 127.27.225.193:46445: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260806 21:10:44.750195 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 16485
I20260806 21:10:44.752310 16790 ts_tablet_manager.cc:1918] T 52eff4fe777c41d2a7872d52b8d9cddc P 2f9fc6cc4ee84efa9ac409aa052eeb1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:10:44.752442 16790 log.cc:1196] T 52eff4fe777c41d2a7872d52b8d9cddc P 2f9fc6cc4ee84efa9ac409aa052eeb1b: Deleting WAL directory at /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050378084738-28551-0/minicluster-data/ts-2/wal/wals/52eff4fe777c41d2a7872d52b8d9cddc
I20260806 21:10:44.752728 16790 ts_tablet_manager.cc:1939] T 52eff4fe777c41d2a7872d52b8d9cddc P 2f9fc6cc4ee84efa9ac409aa052eeb1b: Deleting consensus metadata
I20260806 21:10:44.753971 16273 catalog_manager.cc:4841] TS 2f9fc6cc4ee84efa9ac409aa052eeb1b (127.27.225.195:34397): tablet 52eff4fe777c41d2a7872d52b8d9cddc (table test-table [id=179b874c78a74bb78d4695e403b87982]) successfully deleted
I20260806 21:10:44.757917 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 16617
I20260806 21:10:44.764222 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 16258
2026-08-06T21:10:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1049 ms)
[----------] 6 tests from EncryptionPolicyTest (11267 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:10:44 dist-test-slave-69rg krb5kdc[16797](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:10:44 dist-test-slave-69rg krb5kdc[16797](info): set up 2 sockets
Aug 06 21:10:44 dist-test-slave-69rg krb5kdc[16797](info): commencing operation
krb5kdc: starting...
W20260806 21:10:45.837626 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:10:45 dist-test-slave-69rg krb5kdc[16797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:10:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:10:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 21:10:45.986357 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40551
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:10:46.131686 16810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:46.132011 16810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:46.132057 16810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:46.136339 16810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:10:46.136420 16810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:46.136440 16810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:46.136457 16810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:10:46.136472 16810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:10:46.142510 16810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40551
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:46.143679 16810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:46.144701 16810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:46.154269 16816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:46.154392 16810 server_base.cc:1034] running on GCE node
W20260806 21:10:46.154273 16818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:46.154273 16815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:46.155256 16810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:46.156539 16810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:46.157755 16810 hybrid_clock.cc:648] HybridClock initialized: now 1786050646157720 us; error 53 us; skew 500 ppm
Aug 06 21:10:46 dist-test-slave-69rg krb5kdc[16797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050646, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:46.160900 16810 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 21:10:46.162086 16810 webserver.cc:460] Webserver started at http://127.0.0.1:44813/ using document root <none> and password file <none>
I20260806 21:10:46.162738 16810 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:46.162842 16810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:46.163094 16810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:46.165294 16810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "f17a0d7257d841bebee36a55b44f2c0d"
format_stamp: "Formatted at 2026-08-06 21:10:46 on dist-test-slave-69rg"
server_key: "10e0202d471ec07a24138480ebaf0063"
server_key_iv: "014369cb3e9162dd64329af48b309ea4"
server_key_version: "encryptionkey@0"
I20260806 21:10:46.165979 16810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "f17a0d7257d841bebee36a55b44f2c0d"
format_stamp: "Formatted at 2026-08-06 21:10:46 on dist-test-slave-69rg"
server_key: "10e0202d471ec07a24138480ebaf0063"
server_key_iv: "014369cb3e9162dd64329af48b309ea4"
server_key_version: "encryptionkey@0"
I20260806 21:10:46.170043 16810 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:10:46.172708 16825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:46.174238 16810 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:10:46.174387 16810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "f17a0d7257d841bebee36a55b44f2c0d"
format_stamp: "Formatted at 2026-08-06 21:10:46 on dist-test-slave-69rg"
server_key: "10e0202d471ec07a24138480ebaf0063"
server_key_iv: "014369cb3e9162dd64329af48b309ea4"
server_key_version: "encryptionkey@0"
I20260806 21:10:46.174513 16810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:46.201719 16810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:46.205408 16810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:46.205644 16810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:46.213613 16810 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40551
I20260806 21:10:46.213650 16880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40551 every 8 connection(s)
I20260806 21:10:46.215029 16810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:10:46.216146 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 16810
I20260806 21:10:46.216325 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:10:46.216660 28551 external_mini_cluster.cc:1428] Setting key 3aca0a076d34ea500e39aeaac1852a49
I20260806 21:10:46.218931 16881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:46.226047 16881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f17a0d7257d841bebee36a55b44f2c0d: Bootstrap starting.
Aug 06 21:10:46 dist-test-slave-69rg krb5kdc[16797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 21:10:46.228843 16881 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f17a0d7257d841bebee36a55b44f2c0d: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:46.229616 16881 log.cc:824] T 00000000000000000000000000000000 P f17a0d7257d841bebee36a55b44f2c0d: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:46.231815 16881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f17a0d7257d841bebee36a55b44f2c0d: No bootstrap required, opened a new log
I20260806 21:10:46.232764 16884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:46.218734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35587 (local address 127.0.0.1:40551)
0806 21:10:46.219286 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:46.219318 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:46.219356 (+ 38us) server_negotiation.cc:407] Connection header received
0806 21:10:46.220066 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:46.220084 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:46.220539 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:46.220915 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:46.221783 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:46.222869 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:46.223812 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:46.224322 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:46.224503 (+ 181us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:46.227358 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:46.227363 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:46.227377 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:10:46.227409 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:46.229464 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:46.230204 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:46.230210 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:46.230215 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:46.230340 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:46.230725 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:46.230728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:46.230729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:46.231140 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:46.231382 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:46.231725 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 21:10:46.232002 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":172,"threads_started":1}
I20260806 21:10:46.236263 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:10:46.236331 16881 raft_consensus.cc:348] T 00000000000000000000000000000000 P f17a0d7257d841bebee36a55b44f2c0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17a0d7257d841bebee36a55b44f2c0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40551 } }
I20260806 21:10:46.236629 16881 raft_consensus.cc:374] T 00000000000000000000000000000000 P f17a0d7257d841bebee36a55b44f2c0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:46.236690 16881 raft_consensus.cc:729] T 00000000000000000000000000000000 P f17a0d7257d841bebee36a55b44f2c0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f17a0d7257d841bebee36a55b44f2c0d, State: Initialized, Role: FOLLOWER
I20260806 21:10:46.237185 16881 consensus_queue.cc:260] T 00000000000000000000000000000000 P f17a0d7257d841bebee36a55b44f2c0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17a0d7257d841bebee36a55b44f2c0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40551 } }
I20260806 21:10:46.237324 16881 raft_consensus.cc:388] T 00000000000000000000000000000000 P f17a0d7257d841bebee36a55b44f2c0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:10:46.237412 16881 raft_consensus.cc:482] T 00000000000000000000000000000000 P f17a0d7257d841bebee36a55b44f2c0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:10:46.237514 16881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f17a0d7257d841bebee36a55b44f2c0d [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:46.239369 16881 raft_consensus.cc:504] T 00000000000000000000000000000000 P f17a0d7257d841bebee36a55b44f2c0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17a0d7257d841bebee36a55b44f2c0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40551 } }
I20260806 21:10:46.239820 16881 leader_election.cc:302] T 00000000000000000000000000000000 P f17a0d7257d841bebee36a55b44f2c0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f17a0d7257d841bebee36a55b44f2c0d; no voters:
I20260806 21:10:46.240202 16881 leader_election.cc:288] T 00000000000000000000000000000000 P f17a0d7257d841bebee36a55b44f2c0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:10:46.240343 16891 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f17a0d7257d841bebee36a55b44f2c0d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:10:46.240597 16891 raft_consensus.cc:686] T 00000000000000000000000000000000 P f17a0d7257d841bebee36a55b44f2c0d [term 1 LEADER]: Becoming Leader. State: Replica: f17a0d7257d841bebee36a55b44f2c0d, State: Running, Role: LEADER
I20260806 21:10:46.241001 16891 consensus_queue.cc:237] T 00000000000000000000000000000000 P f17a0d7257d841bebee36a55b44f2c0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17a0d7257d841bebee36a55b44f2c0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40551 } }
I20260806 21:10:46.241644 16881 sys_catalog.cc:564] T 00000000000000000000000000000000 P f17a0d7257d841bebee36a55b44f2c0d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:10:46.242328 16891 sys_catalog.cc:455] T 00000000000000000000000000000000 P f17a0d7257d841bebee36a55b44f2c0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f17a0d7257d841bebee36a55b44f2c0d. Latest consensus state: current_term: 1 leader_uuid: "f17a0d7257d841bebee36a55b44f2c0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17a0d7257d841bebee36a55b44f2c0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40551 } } }
I20260806 21:10:46.242452 16891 sys_catalog.cc:458] T 00000000000000000000000000000000 P f17a0d7257d841bebee36a55b44f2c0d [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:46.243103 16893 sys_catalog.cc:455] T 00000000000000000000000000000000 P f17a0d7257d841bebee36a55b44f2c0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f17a0d7257d841bebee36a55b44f2c0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f17a0d7257d841bebee36a55b44f2c0d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40551 } } }
I20260806 21:10:46.243211 16893 sys_catalog.cc:458] T 00000000000000000000000000000000 P f17a0d7257d841bebee36a55b44f2c0d [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:46.243500 16898 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:10:46.247313 16898 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:10:46.249380 16884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:46.239010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52833 (local address 127.0.0.1:40551)
0806 21:10:46.239146 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:46.239151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:46.239443 (+ 292us) server_negotiation.cc:407] Connection header received
0806 21:10:46.239610 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:46.239613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:46.239670 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:46.239713 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:46.240799 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:46.241573 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:46.242799 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:46.243234 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:46.243320 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:46.245302 (+ 1982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:46.245306 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:46.245309 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:10:46.245337 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:46.247180 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:46.247844 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:46.247848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:46.247850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:46.247943 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:46.248380 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:46.248383 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:46.248385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:46.248621 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:46.248776 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:46.249112 (+ 336us) server_negotiation.cc:299] Negotiation successful
0806 21:10:46.249228 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 21:10:46.254043 16898 catalog_manager.cc:1347] Generated new cluster ID: 87482229c0ad438d919021790a08c1af
I20260806 21:10:46.254141 16898 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:10:46.269138 16898 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:10:46.270275 16898 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:10:46.284691 16898 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f17a0d7257d841bebee36a55b44f2c0d: Generated new TSK 0
I20260806 21:10:46.285519 16898 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:10:46.290107 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 16810
W20260806 21:10:46.297025 16795 connection.cc:535] client connection to 127.0.0.1:40551 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40551): Connection reset by peer (error 104)
2026-08-06T21:10:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1524 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:10:46 dist-test-slave-69rg krb5kdc[16912](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:10:46 dist-test-slave-69rg krb5kdc[16912](info): set up 2 sockets
Aug 06 21:10:46 dist-test-slave-69rg krb5kdc[16912](info): commencing operation
krb5kdc: starting...
W20260806 21:10:47.369746 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:10:47 dist-test-slave-69rg krb5kdc[16912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:10:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:10:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 21:10:47.507535 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38837
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:10:47.621271 16926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:47.621558 16926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:47.621603 16926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:47.625542 16926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:10:47.625605 16926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:47.625624 16926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:47.625641 16926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:10:47.625658 16926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:10:47.631049 16926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38837
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:47.632136 16926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:47.633128 16926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:47.642201 16931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:47.642259 16932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:47.642372 16926 server_base.cc:1034] running on GCE node
W20260806 21:10:47.642580 16934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:47.643289 16926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:47.644567 16926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:47.645785 16926 hybrid_clock.cc:648] HybridClock initialized: now 1786050647645763 us; error 50 us; skew 500 ppm
Aug 06 21:10:47 dist-test-slave-69rg krb5kdc[16912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050647, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:47.653417 16926 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 21:10:47.654670 16926 webserver.cc:460] Webserver started at http://127.0.0.1:43077/ using document root <none> and password file <none>
I20260806 21:10:47.655295 16926 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:47.655395 16926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:47.655650 16926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:47.657872 16926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "9c84e320c13d4a349f9715867c411228"
format_stamp: "Formatted at 2026-08-06 21:10:47 on dist-test-slave-69rg"
server_key: "4c7788dad6e6ecee72e8b439fc1569c4"
server_key_iv: "1f304cdae5cddb320f351eaf85b43054"
server_key_version: "encryptionkey@0"
I20260806 21:10:47.658506 16926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "9c84e320c13d4a349f9715867c411228"
format_stamp: "Formatted at 2026-08-06 21:10:47 on dist-test-slave-69rg"
server_key: "4c7788dad6e6ecee72e8b439fc1569c4"
server_key_iv: "1f304cdae5cddb320f351eaf85b43054"
server_key_version: "encryptionkey@0"
I20260806 21:10:47.662463 16926 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 21:10:47.665037 16942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:47.666349 16926 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:10:47.666493 16926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "9c84e320c13d4a349f9715867c411228"
format_stamp: "Formatted at 2026-08-06 21:10:47 on dist-test-slave-69rg"
server_key: "4c7788dad6e6ecee72e8b439fc1569c4"
server_key_iv: "1f304cdae5cddb320f351eaf85b43054"
server_key_version: "encryptionkey@0"
I20260806 21:10:47.666612 16926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:47.697266 16926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:47.700543 16926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:47.700765 16926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:47.708663 16926 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38837
I20260806 21:10:47.708690 16995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38837 every 8 connection(s)
I20260806 21:10:47.709652 16926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:10:47.712633 16996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:47.715768 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 16926
I20260806 21:10:47.715937 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:10:47.716231 28551 external_mini_cluster.cc:1428] Setting key 665da2f0fcccc6c458c29e13d63f43ee
I20260806 21:10:47.718928 16996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c84e320c13d4a349f9715867c411228: Bootstrap starting.
I20260806 21:10:47.721670 16996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c84e320c13d4a349f9715867c411228: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:47.722522 16996 log.cc:824] T 00000000000000000000000000000000 P 9c84e320c13d4a349f9715867c411228: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:47.724797 16996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c84e320c13d4a349f9715867c411228: No bootstrap required, opened a new log
I20260806 21:10:47.726846 16999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:47.717718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32875 (local address 127.0.0.1:38837)
0806 21:10:47.718218 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:47.718239 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:47.718281 (+ 42us) server_negotiation.cc:407] Connection header received
0806 21:10:47.719137 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:47.719162 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:47.719588 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:47.719960 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:47.720404 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:47.721910 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:47.723011 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:47.723577 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:47.723741 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:47.724778 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:47.724785 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:47.724800 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:47.725296 (+ 496us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:47.725363 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:47.725394 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:47.725533 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:47.725755 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 21:10:47.725929 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":114,"threads_started":1}
I20260806 21:10:47.729312 16996 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c84e320c13d4a349f9715867c411228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c84e320c13d4a349f9715867c411228" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38837 } }
I20260806 21:10:47.729602 16996 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c84e320c13d4a349f9715867c411228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:47.729657 16996 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c84e320c13d4a349f9715867c411228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c84e320c13d4a349f9715867c411228, State: Initialized, Role: FOLLOWER
I20260806 21:10:47.730106 16996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c84e320c13d4a349f9715867c411228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c84e320c13d4a349f9715867c411228" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38837 } }
I20260806 21:10:47.730314 16996 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c84e320c13d4a349f9715867c411228 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:10:47.730388 16996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c84e320c13d4a349f9715867c411228 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:10:47.730389 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:10:47.730489 16996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c84e320c13d4a349f9715867c411228 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:47.731527 16996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c84e320c13d4a349f9715867c411228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c84e320c13d4a349f9715867c411228" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38837 } }
I20260806 21:10:47.731945 16996 leader_election.cc:302] T 00000000000000000000000000000000 P 9c84e320c13d4a349f9715867c411228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c84e320c13d4a349f9715867c411228; no voters:
I20260806 21:10:47.732328 16996 leader_election.cc:288] T 00000000000000000000000000000000 P 9c84e320c13d4a349f9715867c411228 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:10:47.732409 17005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c84e320c13d4a349f9715867c411228 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:10:47.732618 17005 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c84e320c13d4a349f9715867c411228 [term 1 LEADER]: Becoming Leader. State: Replica: 9c84e320c13d4a349f9715867c411228, State: Running, Role: LEADER
I20260806 21:10:47.732952 17005 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c84e320c13d4a349f9715867c411228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c84e320c13d4a349f9715867c411228" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38837 } }
I20260806 21:10:47.733764 16996 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c84e320c13d4a349f9715867c411228 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:10:47.734220 17007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c84e320c13d4a349f9715867c411228 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c84e320c13d4a349f9715867c411228. Latest consensus state: current_term: 1 leader_uuid: "9c84e320c13d4a349f9715867c411228" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c84e320c13d4a349f9715867c411228" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38837 } } }
I20260806 21:10:47.734344 17007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c84e320c13d4a349f9715867c411228 [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:47.735038 17006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c84e320c13d4a349f9715867c411228 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c84e320c13d4a349f9715867c411228" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c84e320c13d4a349f9715867c411228" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38837 } } }
I20260806 21:10:47.735136 17006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c84e320c13d4a349f9715867c411228 [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:47.735399 17011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:10:47.738242 16999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:47.732673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44173 (local address 127.0.0.1:38837)
0806 21:10:47.732810 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:47.732818 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:47.733155 (+ 337us) server_negotiation.cc:407] Connection header received
0806 21:10:47.733354 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:47.733359 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:47.733414 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:47.733491 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:47.733970 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:47.734743 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:47.736041 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:47.736484 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:47.736592 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:47.737656 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:47.737662 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:47.737665 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:47.737679 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:47.737710 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:47.737723 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:47.737847 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:47.738042 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 21:10:47.738092 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 21:10:47.739312 17011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:10:47.744740 17011 catalog_manager.cc:1347] Generated new cluster ID: 09fd6fc9329d4988b022a64747589a6b
I20260806 21:10:47.744822 17011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:10:47.776449 17011 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:10:47.777488 17011 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:10:47.784623 17011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c84e320c13d4a349f9715867c411228: Generated new TSK 0
I20260806 21:10:47.785358 17011 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:10:47.797362 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 16926
W20260806 21:10:47.803817 16910 connection.cc:535] client connection to 127.0.0.1:38837 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38837): Connection reset by peer (error 104)
2026-08-06T21:10:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1506 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:10:47 dist-test-slave-69rg krb5kdc[17028](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:10:47 dist-test-slave-69rg krb5kdc[17028](info): set up 2 sockets
Aug 06 21:10:47 dist-test-slave-69rg krb5kdc[17028](info): commencing operation
krb5kdc: starting...
W20260806 21:10:48.862404 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:10:48 dist-test-slave-69rg krb5kdc[17028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:10:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:10:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 21:10:49.010946 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35185
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:10:49.127014 17041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:49.127355 17041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:49.127430 17041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:49.131532 17041 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:10:49.131657 17041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:49.131698 17041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:49.131758 17041 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:10:49.131800 17041 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:10:49.137183 17041 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35185
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:49.138425 17041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:49.139467 17041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:49.148439 17049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:49.148448 17046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:49.148460 17047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:49.148784 17041 server_base.cc:1034] running on GCE node
I20260806 21:10:49.149415 17041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:49.150687 17041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:49.151914 17041 hybrid_clock.cc:648] HybridClock initialized: now 1786050649151874 us; error 58 us; skew 500 ppm
Aug 06 21:10:49 dist-test-slave-69rg krb5kdc[17028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050649, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:49.155109 17041 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 21:10:49.156169 17041 webserver.cc:460] Webserver started at http://127.0.0.1:45007/ using document root <none> and password file <none>
I20260806 21:10:49.156790 17041 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:49.156888 17041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:49.157145 17041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:49.159459 17041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "fab783404a75453ea2961b803868d620"
format_stamp: "Formatted at 2026-08-06 21:10:49 on dist-test-slave-69rg"
server_key: "2a122cfeb41989111cafcf78faff98e7"
server_key_iv: "fe7ca1cd6080d3dc5b34e7ceb4a0d26e"
server_key_version: "encryptionkey@0"
I20260806 21:10:49.160068 17041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "fab783404a75453ea2961b803868d620"
format_stamp: "Formatted at 2026-08-06 21:10:49 on dist-test-slave-69rg"
server_key: "2a122cfeb41989111cafcf78faff98e7"
server_key_iv: "fe7ca1cd6080d3dc5b34e7ceb4a0d26e"
server_key_version: "encryptionkey@0"
I20260806 21:10:49.164474 17041 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:10:49.167359 17056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:49.168774 17041 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 21:10:49.168946 17041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "fab783404a75453ea2961b803868d620"
format_stamp: "Formatted at 2026-08-06 21:10:49 on dist-test-slave-69rg"
server_key: "2a122cfeb41989111cafcf78faff98e7"
server_key_iv: "fe7ca1cd6080d3dc5b34e7ceb4a0d26e"
server_key_version: "encryptionkey@0"
I20260806 21:10:49.169082 17041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:49.174079 17041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:49.177470 17041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:49.177695 17041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:49.185884 17041 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35185
I20260806 21:10:49.185920 17110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35185 every 8 connection(s)
I20260806 21:10:49.186878 17041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:10:49.189170 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 17041
I20260806 21:10:49.189340 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:10:49.189661 28551 external_mini_cluster.cc:1428] Setting key 003806d49e33a33b3685e552d0d5b2cd
I20260806 21:10:49.190086 17111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:49.196060 17114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:49.191275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44839 (local address 127.0.0.1:35185)
0806 21:10:49.191986 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:49.192003 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:49.192051 (+ 48us) server_negotiation.cc:407] Connection header received
0806 21:10:49.193159 (+ 1108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:49.193182 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:49.193488 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:49.193914 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:49.194784 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:49.194789 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:49.194801 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:49.194813 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:49.194870 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:49.194890 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:49.195053 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:49.195135 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 21:10:49.195303 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":169,"threads_started":1}
I20260806 21:10:49.197538 17111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fab783404a75453ea2961b803868d620: Bootstrap starting.
I20260806 21:10:49.199230 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:10:49.200347 17111 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fab783404a75453ea2961b803868d620: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:49.201228 17111 log.cc:824] T 00000000000000000000000000000000 P fab783404a75453ea2961b803868d620: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:49.203423 17114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:49.201495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40829 (local address 127.0.0.1:35185)
0806 21:10:49.201655 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:49.201660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:49.201812 (+ 152us) server_negotiation.cc:407] Connection header received
0806 21:10:49.201995 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:49.201998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:49.202040 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:49.202127 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:49.202918 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:49.202922 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:49.202925 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:49.202938 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:49.202972 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:49.202984 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:49.203082 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:49.203247 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 21:10:49.203295 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 21:10:49.204033 17111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fab783404a75453ea2961b803868d620: No bootstrap required, opened a new log
I20260806 21:10:49.208473 17111 raft_consensus.cc:348] T 00000000000000000000000000000000 P fab783404a75453ea2961b803868d620 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fab783404a75453ea2961b803868d620" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35185 } }
I20260806 21:10:49.208781 17111 raft_consensus.cc:374] T 00000000000000000000000000000000 P fab783404a75453ea2961b803868d620 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:49.208875 17111 raft_consensus.cc:729] T 00000000000000000000000000000000 P fab783404a75453ea2961b803868d620 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fab783404a75453ea2961b803868d620, State: Initialized, Role: FOLLOWER
I20260806 21:10:49.209367 17111 consensus_queue.cc:260] T 00000000000000000000000000000000 P fab783404a75453ea2961b803868d620 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fab783404a75453ea2961b803868d620" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35185 } }
I20260806 21:10:49.209542 17111 raft_consensus.cc:388] T 00000000000000000000000000000000 P fab783404a75453ea2961b803868d620 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:10:49.209687 17111 raft_consensus.cc:482] T 00000000000000000000000000000000 P fab783404a75453ea2961b803868d620 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:10:49.209825 17111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fab783404a75453ea2961b803868d620 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:49.210994 17111 raft_consensus.cc:504] T 00000000000000000000000000000000 P fab783404a75453ea2961b803868d620 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fab783404a75453ea2961b803868d620" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35185 } }
I20260806 21:10:49.211396 17111 leader_election.cc:302] T 00000000000000000000000000000000 P fab783404a75453ea2961b803868d620 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fab783404a75453ea2961b803868d620; no voters:
I20260806 21:10:49.211892 17111 leader_election.cc:288] T 00000000000000000000000000000000 P fab783404a75453ea2961b803868d620 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:10:49.212073 17121 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fab783404a75453ea2961b803868d620 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:10:49.212395 17121 raft_consensus.cc:686] T 00000000000000000000000000000000 P fab783404a75453ea2961b803868d620 [term 1 LEADER]: Becoming Leader. State: Replica: fab783404a75453ea2961b803868d620, State: Running, Role: LEADER
I20260806 21:10:49.212787 17121 consensus_queue.cc:237] T 00000000000000000000000000000000 P fab783404a75453ea2961b803868d620 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fab783404a75453ea2961b803868d620" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35185 } }
I20260806 21:10:49.213604 17111 sys_catalog.cc:564] T 00000000000000000000000000000000 P fab783404a75453ea2961b803868d620 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:10:49.214604 17122 sys_catalog.cc:455] T 00000000000000000000000000000000 P fab783404a75453ea2961b803868d620 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fab783404a75453ea2961b803868d620" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fab783404a75453ea2961b803868d620" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35185 } } }
I20260806 21:10:49.214805 17122 sys_catalog.cc:458] T 00000000000000000000000000000000 P fab783404a75453ea2961b803868d620 [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:49.214908 17123 sys_catalog.cc:455] T 00000000000000000000000000000000 P fab783404a75453ea2961b803868d620 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fab783404a75453ea2961b803868d620. Latest consensus state: current_term: 1 leader_uuid: "fab783404a75453ea2961b803868d620" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fab783404a75453ea2961b803868d620" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35185 } } }
I20260806 21:10:49.214995 17123 sys_catalog.cc:458] T 00000000000000000000000000000000 P fab783404a75453ea2961b803868d620 [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:49.215322 17130 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:10:49.218801 17130 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:10:49.225114 17130 catalog_manager.cc:1347] Generated new cluster ID: 54e9fa5bfdf04213a1173229379085cb
I20260806 21:10:49.225324 17130 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:10:49.243467 17130 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:10:49.243834 17130 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:10:49.255050 17130 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fab783404a75453ea2961b803868d620: Generated new TSK 0
I20260806 21:10:49.256001 17130 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:10:49.261508 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 17041
2026-08-06T21:10:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1465 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:10:49 dist-test-slave-69rg krb5kdc[17142](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:10:49 dist-test-slave-69rg krb5kdc[17142](info): set up 2 sockets
Aug 06 21:10:49 dist-test-slave-69rg krb5kdc[17142](info): commencing operation
krb5kdc: starting...
W20260806 21:10:50.321764 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:10:50 dist-test-slave-69rg krb5kdc[17142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:10:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:10:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 21:10:50.462455 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42287
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42287
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:10:50.576875 17155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:50.577193 17155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:50.577287 17155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:50.581573 17155 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:10:50.581673 17155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:50.581709 17155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:50.581770 17155 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:10:50.581841 17155 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:10:50.587476 17155 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:50.588596 17155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:50.589704 17155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:50.599488 17163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:50.599530 17160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:50.599714 17161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:50.600525 17155 server_base.cc:1034] running on GCE node
I20260806 21:10:50.601327 17155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:50.603071 17155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:50.604321 17155 hybrid_clock.cc:648] HybridClock initialized: now 1786050650604291 us; error 43 us; skew 500 ppm
Aug 06 21:10:50 dist-test-slave-69rg krb5kdc[17142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050650, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:50.607542 17155 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 21:10:50.608479 17155 webserver.cc:460] Webserver started at http://127.0.0.1:35969/ using document root <none> and password file <none>
I20260806 21:10:50.609066 17155 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:50.609129 17155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:50.609380 17155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:50.611635 17155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "70cf39d121144992b830e65c35e66b28"
format_stamp: "Formatted at 2026-08-06 21:10:50 on dist-test-slave-69rg"
server_key: "5f73d0605259a29171a1d5075c24f3b2"
server_key_iv: "fe2b1cccefe71718b936735b5800b775"
server_key_version: "encryptionkey@0"
I20260806 21:10:50.612226 17155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "70cf39d121144992b830e65c35e66b28"
format_stamp: "Formatted at 2026-08-06 21:10:50 on dist-test-slave-69rg"
server_key: "5f73d0605259a29171a1d5075c24f3b2"
server_key_iv: "fe2b1cccefe71718b936735b5800b775"
server_key_version: "encryptionkey@0"
I20260806 21:10:50.616096 17155 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 21:10:50.618618 17170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:50.619954 17155 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 21:10:50.620091 17155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "70cf39d121144992b830e65c35e66b28"
format_stamp: "Formatted at 2026-08-06 21:10:50 on dist-test-slave-69rg"
server_key: "5f73d0605259a29171a1d5075c24f3b2"
server_key_iv: "fe2b1cccefe71718b936735b5800b775"
server_key_version: "encryptionkey@0"
I20260806 21:10:50.620206 17155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:50.659576 17155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:50.662730 17155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:50.662946 17155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:50.670276 17155 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42287
I20260806 21:10:50.670305 17223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42287 every 8 connection(s)
I20260806 21:10:50.671293 17155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:10:50.674022 17224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:50.679859 17224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70cf39d121144992b830e65c35e66b28: Bootstrap starting.
I20260806 21:10:50.680974 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 17155
I20260806 21:10:50.681089 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:10:50.681751 28551 external_mini_cluster.cc:1428] Setting key 7559fa4a787388bb5b8bff2d760ed998
I20260806 21:10:50.682896 17224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70cf39d121144992b830e65c35e66b28: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:50.683848 17224 log.cc:824] T 00000000000000000000000000000000 P 70cf39d121144992b830e65c35e66b28: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:50.686301 17224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70cf39d121144992b830e65c35e66b28: No bootstrap required, opened a new log
I20260806 21:10:50.690395 17224 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70cf39d121144992b830e65c35e66b28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70cf39d121144992b830e65c35e66b28" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42287 } }
I20260806 21:10:50.690734 17224 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70cf39d121144992b830e65c35e66b28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:50.690842 17224 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70cf39d121144992b830e65c35e66b28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70cf39d121144992b830e65c35e66b28, State: Initialized, Role: FOLLOWER
I20260806 21:10:50.691311 17224 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70cf39d121144992b830e65c35e66b28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70cf39d121144992b830e65c35e66b28" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42287 } }
I20260806 21:10:50.691437 17224 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70cf39d121144992b830e65c35e66b28 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:10:50.691529 17224 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70cf39d121144992b830e65c35e66b28 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:10:50.691638 17224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70cf39d121144992b830e65c35e66b28 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 21:10:50 dist-test-slave-69rg krb5kdc[17142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 21:10:50.692701 17224 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70cf39d121144992b830e65c35e66b28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70cf39d121144992b830e65c35e66b28" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42287 } }
I20260806 21:10:50.693094 17224 leader_election.cc:302] T 00000000000000000000000000000000 P 70cf39d121144992b830e65c35e66b28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70cf39d121144992b830e65c35e66b28; no voters:
I20260806 21:10:50.693480 17224 leader_election.cc:288] T 00000000000000000000000000000000 P 70cf39d121144992b830e65c35e66b28 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:10:50.693630 17229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70cf39d121144992b830e65c35e66b28 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:10:50.693936 17229 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70cf39d121144992b830e65c35e66b28 [term 1 LEADER]: Becoming Leader. State: Replica: 70cf39d121144992b830e65c35e66b28, State: Running, Role: LEADER
I20260806 21:10:50.694324 17229 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70cf39d121144992b830e65c35e66b28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70cf39d121144992b830e65c35e66b28" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42287 } }
I20260806 21:10:50.694983 17224 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70cf39d121144992b830e65c35e66b28 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:10:50.697094 17231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70cf39d121144992b830e65c35e66b28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70cf39d121144992b830e65c35e66b28. Latest consensus state: current_term: 1 leader_uuid: "70cf39d121144992b830e65c35e66b28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70cf39d121144992b830e65c35e66b28" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42287 } } }
I20260806 21:10:50.697201 17231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70cf39d121144992b830e65c35e66b28 [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:50.697052 17230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70cf39d121144992b830e65c35e66b28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70cf39d121144992b830e65c35e66b28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70cf39d121144992b830e65c35e66b28" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42287 } } }
I20260806 21:10:50.698374 17230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70cf39d121144992b830e65c35e66b28 [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:50.699210 17227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:50.683081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48323 (local address 127.0.0.1:42287)
0806 21:10:50.683595 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:50.683618 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:50.683667 (+ 49us) server_negotiation.cc:407] Connection header received
0806 21:10:50.684563 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:50.684583 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:50.685038 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:50.685437 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:50.686215 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:50.687705 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:50.688609 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:50.689962 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:50.690135 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:50.693098 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:50.693106 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:50.693127 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:10:50.693161 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:50.695387 (+ 2226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:50.695941 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:50.695947 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:50.695953 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:50.696103 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:50.696445 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:50.696449 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:50.696451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:50.697049 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:50.697264 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:50.698056 (+ 792us) server_negotiation.cc:299] Negotiation successful
0806 21:10:50.698422 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":161,"threads_started":1}
W20260806 21:10:50.700851 17243 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70cf39d121144992b830e65c35e66b28: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:10:50.700943 17243 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:10:50.701047 17245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:10:50.702101 17245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:10:50.702538 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:10:50.708376 17245 catalog_manager.cc:1347] Generated new cluster ID: 836c5e62136c449c950033c45d0e6f86
I20260806 21:10:50.708452 17245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:10:50.713967 17227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:50.704816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59999 (local address 127.0.0.1:42287)
0806 21:10:50.704929 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:50.704935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:50.705268 (+ 333us) server_negotiation.cc:407] Connection header received
0806 21:10:50.705408 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:50.705411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:50.705461 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:50.705549 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:50.706317 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:50.707046 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:50.708269 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:50.708619 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:50.708743 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:50.710533 (+ 1790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:50.710537 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:50.710539 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:10:50.710566 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:50.712192 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:50.712690 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:50.712693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:50.712695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:50.712781 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:50.713062 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:50.713064 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:50.713066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:50.713253 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:50.713370 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:50.713623 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 21:10:50.713741 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 21:10:50.740146 17245 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:10:50.741149 17245 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:10:50.747949 17245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70cf39d121144992b830e65c35e66b28: Generated new TSK 0
I20260806 21:10:50.748665 17245 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:10:50.760711 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 17155
2026-08-06T21:10:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1498 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:10:50 dist-test-slave-69rg krb5kdc[17256](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:10:50 dist-test-slave-69rg krb5kdc[17256](info): set up 2 sockets
Aug 06 21:10:50 dist-test-slave-69rg krb5kdc[17256](info): commencing operation
krb5kdc: starting...
W20260806 21:10:51.885517 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:10:51 dist-test-slave-69rg krb5kdc[17256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:10:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:10:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 21:10:52.027087 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43821
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:10:52.161491 17270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:52.161957 17270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:52.162071 17270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:52.166898 17270 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:10:52.166999 17270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:52.167043 17270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:52.167122 17270 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:10:52.167162 17270 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:10:52.173224 17270 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:52.174459 17270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:52.175539 17270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:52.184780 17275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:52.184787 17278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:52.184780 17276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:52.185287 17270 server_base.cc:1034] running on GCE node
I20260806 21:10:52.185947 17270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:52.187151 17270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:52.188340 17270 hybrid_clock.cc:648] HybridClock initialized: now 1786050652188318 us; error 35 us; skew 500 ppm
Aug 06 21:10:52 dist-test-slave-69rg krb5kdc[17256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050652, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:52.191452 17270 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 21:10:52.192417 17270 webserver.cc:460] Webserver started at http://127.0.0.1:41369/ using document root <none> and password file <none>
I20260806 21:10:52.192965 17270 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:52.193024 17270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:52.193194 17270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:52.195363 17270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "782d50ad358f43549b9a052498d68025"
format_stamp: "Formatted at 2026-08-06 21:10:52 on dist-test-slave-69rg"
server_key: "a55424a2bc57be2f75732ddc36781141"
server_key_iv: "064b8a913e76d1485c5b41fafa56f0ba"
server_key_version: "encryptionkey@0"
I20260806 21:10:52.195901 17270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "782d50ad358f43549b9a052498d68025"
format_stamp: "Formatted at 2026-08-06 21:10:52 on dist-test-slave-69rg"
server_key: "a55424a2bc57be2f75732ddc36781141"
server_key_iv: "064b8a913e76d1485c5b41fafa56f0ba"
server_key_version: "encryptionkey@0"
I20260806 21:10:52.199858 17270 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:10:52.202553 17286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:52.203886 17270 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:10:52.204053 17270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "782d50ad358f43549b9a052498d68025"
format_stamp: "Formatted at 2026-08-06 21:10:52 on dist-test-slave-69rg"
server_key: "a55424a2bc57be2f75732ddc36781141"
server_key_iv: "064b8a913e76d1485c5b41fafa56f0ba"
server_key_version: "encryptionkey@0"
I20260806 21:10:52.204190 17270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:52.224406 17270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:52.227602 17270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:52.227841 17270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:52.235520 17270 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43821
I20260806 21:10:52.235538 17339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43821 every 8 connection(s)
I20260806 21:10:52.236555 17270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:10:52.240058 17340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:52.246004 17340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 782d50ad358f43549b9a052498d68025: Bootstrap starting.
I20260806 21:10:52.246112 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 17270
I20260806 21:10:52.246264 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:10:52.246547 28551 external_mini_cluster.cc:1428] Setting key 8f7e0e88967d94055f5907f61c523b6b
I20260806 21:10:52.248622 17340 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 782d50ad358f43549b9a052498d68025: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:52.249377 17340 log.cc:824] T 00000000000000000000000000000000 P 782d50ad358f43549b9a052498d68025: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:52.251505 17340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 782d50ad358f43549b9a052498d68025: No bootstrap required, opened a new log
I20260806 21:10:52.256070 17343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:52.247967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56437 (local address 127.0.0.1:43821)
0806 21:10:52.248390 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:52.248406 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:52.248440 (+ 34us) server_negotiation.cc:407] Connection header received
0806 21:10:52.249139 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:52.249157 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:52.249514 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:52.249918 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:52.250380 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:52.251541 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:52.252518 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:52.253011 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:52.253191 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:52.254177 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:52.254183 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:52.254197 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:52.254617 (+ 420us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:52.254674 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:52.254698 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:52.254812 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:52.254994 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 21:10:52.255242 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":140,"threads_started":1}
I20260806 21:10:52.256059 17340 raft_consensus.cc:348] T 00000000000000000000000000000000 P 782d50ad358f43549b9a052498d68025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "782d50ad358f43549b9a052498d68025" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43821 } }
I20260806 21:10:52.256438 17340 raft_consensus.cc:374] T 00000000000000000000000000000000 P 782d50ad358f43549b9a052498d68025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:52.256527 17340 raft_consensus.cc:729] T 00000000000000000000000000000000 P 782d50ad358f43549b9a052498d68025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 782d50ad358f43549b9a052498d68025, State: Initialized, Role: FOLLOWER
I20260806 21:10:52.257094 17340 consensus_queue.cc:260] T 00000000000000000000000000000000 P 782d50ad358f43549b9a052498d68025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "782d50ad358f43549b9a052498d68025" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43821 } }
I20260806 21:10:52.257289 17340 raft_consensus.cc:388] T 00000000000000000000000000000000 P 782d50ad358f43549b9a052498d68025 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:10:52.257351 17340 raft_consensus.cc:482] T 00000000000000000000000000000000 P 782d50ad358f43549b9a052498d68025 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:10:52.257455 17340 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 782d50ad358f43549b9a052498d68025 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:52.258927 17340 raft_consensus.cc:504] T 00000000000000000000000000000000 P 782d50ad358f43549b9a052498d68025 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "782d50ad358f43549b9a052498d68025" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43821 } }
I20260806 21:10:52.259370 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:10:52.259322 17340 leader_election.cc:302] T 00000000000000000000000000000000 P 782d50ad358f43549b9a052498d68025 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 782d50ad358f43549b9a052498d68025; no voters:
I20260806 21:10:52.259698 17340 leader_election.cc:288] T 00000000000000000000000000000000 P 782d50ad358f43549b9a052498d68025 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:10:52.259832 17345 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 782d50ad358f43549b9a052498d68025 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:10:52.260092 17345 raft_consensus.cc:686] T 00000000000000000000000000000000 P 782d50ad358f43549b9a052498d68025 [term 1 LEADER]: Becoming Leader. State: Replica: 782d50ad358f43549b9a052498d68025, State: Running, Role: LEADER
I20260806 21:10:52.260511 17345 consensus_queue.cc:237] T 00000000000000000000000000000000 P 782d50ad358f43549b9a052498d68025 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "782d50ad358f43549b9a052498d68025" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43821 } }
I20260806 21:10:52.261107 17340 sys_catalog.cc:564] T 00000000000000000000000000000000 P 782d50ad358f43549b9a052498d68025 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:10:52.262337 17346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 782d50ad358f43549b9a052498d68025 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "782d50ad358f43549b9a052498d68025" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "782d50ad358f43549b9a052498d68025" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43821 } } }
I20260806 21:10:52.262495 17346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 782d50ad358f43549b9a052498d68025 [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:52.262916 17359 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:10:52.262974 17347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 782d50ad358f43549b9a052498d68025 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 782d50ad358f43549b9a052498d68025. Latest consensus state: current_term: 1 leader_uuid: "782d50ad358f43549b9a052498d68025" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "782d50ad358f43549b9a052498d68025" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43821 } } }
I20260806 21:10:52.263193 17347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 782d50ad358f43549b9a052498d68025 [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:52.266595 17359 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:10:52.269441 17343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:52.262049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54159 (local address 127.0.0.1:43821)
0806 21:10:52.262158 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:52.262163 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:52.264119 (+ 1956us) server_negotiation.cc:407] Connection header received
0806 21:10:52.264248 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:52.264251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:52.264305 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:52.264414 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:52.264767 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:52.265529 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:52.266979 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:52.267362 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:52.267456 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:52.268632 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:52.268636 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:52.268640 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:52.268659 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:52.268700 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:52.268719 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:52.268805 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:52.269209 (+ 404us) server_negotiation.cc:299] Negotiation successful
0806 21:10:52.269278 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 21:10:52.274008 17359 catalog_manager.cc:1347] Generated new cluster ID: 2f9a8d0ee0bd4266aba3e7e56abf2386
I20260806 21:10:52.274113 17359 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:10:52.295313 17359 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:10:52.296367 17359 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:10:52.309064 17359 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 782d50ad358f43549b9a052498d68025: Generated new TSK 0
I20260806 21:10:52.309880 17359 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:10:52.318856 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 17270
2026-08-06T21:10:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1558 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:10:52 dist-test-slave-69rg krb5kdc[17371](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:10:52 dist-test-slave-69rg krb5kdc[17371](info): set up 2 sockets
Aug 06 21:10:52 dist-test-slave-69rg krb5kdc[17371](info): commencing operation
krb5kdc: starting...
W20260806 21:10:53.389402 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:10:53 dist-test-slave-69rg krb5kdc[17371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:10:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:10:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 21:10:53.524729 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42629
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:10:53.638594 17385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:53.638899 17385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:53.638986 17385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:53.643126 17385 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:10:53.643222 17385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:53.643260 17385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:53.643316 17385 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:10:53.643355 17385 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:10:53.648736 17385 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:53.649942 17385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:53.650952 17385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:53.659678 17391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:53.659689 17390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:53.659704 17393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:53.659948 17385 server_base.cc:1034] running on GCE node
I20260806 21:10:53.660694 17385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:53.661908 17385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:53.663096 17385 hybrid_clock.cc:648] HybridClock initialized: now 1786050653663076 us; error 39 us; skew 500 ppm
Aug 06 21:10:53 dist-test-slave-69rg krb5kdc[17371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050653, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:53.671633 17385 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 21:10:53.672634 17385 webserver.cc:460] Webserver started at http://127.0.0.1:46779/ using document root <none> and password file <none>
I20260806 21:10:53.673221 17385 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:53.673321 17385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:53.673553 17385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:53.675709 17385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "dddf2ff4a77a4e71a9473a1975529123"
format_stamp: "Formatted at 2026-08-06 21:10:53 on dist-test-slave-69rg"
server_key: "a3b3a85a41acd917fafaa090a990e724"
server_key_iv: "c975b4a3c793f3c92cd107a6dbc62100"
server_key_version: "encryptionkey@0"
I20260806 21:10:53.676271 17385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "dddf2ff4a77a4e71a9473a1975529123"
format_stamp: "Formatted at 2026-08-06 21:10:53 on dist-test-slave-69rg"
server_key: "a3b3a85a41acd917fafaa090a990e724"
server_key_iv: "c975b4a3c793f3c92cd107a6dbc62100"
server_key_version: "encryptionkey@0"
I20260806 21:10:53.680198 17385 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:10:53.682850 17401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:53.684113 17385 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260806 21:10:53.684267 17385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "dddf2ff4a77a4e71a9473a1975529123"
format_stamp: "Formatted at 2026-08-06 21:10:53 on dist-test-slave-69rg"
server_key: "a3b3a85a41acd917fafaa090a990e724"
server_key_iv: "c975b4a3c793f3c92cd107a6dbc62100"
server_key_version: "encryptionkey@0"
I20260806 21:10:53.684412 17385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:53.689158 17385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:53.692413 17385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:53.692610 17385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:53.699990 17385 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42629
I20260806 21:10:53.700016 17454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42629 every 8 connection(s)
I20260806 21:10:53.700971 17385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:10:53.702257 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 17385
I20260806 21:10:53.702406 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:10:53.702703 28551 external_mini_cluster.cc:1428] Setting key 899982706b86f33dd0d08aba83bacd0e
I20260806 21:10:53.704157 17455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:53.707927 17458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:53.704200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58831 (local address 127.0.0.1:42629)
0806 21:10:53.704628 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:53.704644 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:53.704677 (+ 33us) server_negotiation.cc:407] Connection header received
0806 21:10:53.705335 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:53.705351 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:53.705620 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:53.705984 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:53.706779 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:53.706784 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:53.706797 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:53.706807 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:53.706859 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:53.706878 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:53.707027 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:53.707097 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 21:10:53.707247 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":140,"threads_started":1}
I20260806 21:10:53.710494 17455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dddf2ff4a77a4e71a9473a1975529123: Bootstrap starting.
I20260806 21:10:53.710839 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:10:53.712807 17455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dddf2ff4a77a4e71a9473a1975529123: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:53.713726 17455 log.cc:824] T 00000000000000000000000000000000 P dddf2ff4a77a4e71a9473a1975529123: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:53.714673 17458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:53.712890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40557 (local address 127.0.0.1:42629)
0806 21:10:53.713010 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:53.713015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:53.713203 (+ 188us) server_negotiation.cc:407] Connection header received
0806 21:10:53.713337 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:53.713339 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:53.713370 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:53.713435 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:53.714249 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:53.714252 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:53.714255 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:53.714267 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:53.714322 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:53.714343 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:53.714434 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:53.714500 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 21:10:53.714542 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 21:10:53.715986 17455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dddf2ff4a77a4e71a9473a1975529123: No bootstrap required, opened a new log
I20260806 21:10:53.719694 17455 raft_consensus.cc:348] T 00000000000000000000000000000000 P dddf2ff4a77a4e71a9473a1975529123 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dddf2ff4a77a4e71a9473a1975529123" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42629 } }
I20260806 21:10:53.719990 17455 raft_consensus.cc:374] T 00000000000000000000000000000000 P dddf2ff4a77a4e71a9473a1975529123 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:53.720080 17455 raft_consensus.cc:729] T 00000000000000000000000000000000 P dddf2ff4a77a4e71a9473a1975529123 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dddf2ff4a77a4e71a9473a1975529123, State: Initialized, Role: FOLLOWER
I20260806 21:10:53.720602 17455 consensus_queue.cc:260] T 00000000000000000000000000000000 P dddf2ff4a77a4e71a9473a1975529123 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dddf2ff4a77a4e71a9473a1975529123" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42629 } }
I20260806 21:10:53.720760 17455 raft_consensus.cc:388] T 00000000000000000000000000000000 P dddf2ff4a77a4e71a9473a1975529123 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:10:53.720850 17455 raft_consensus.cc:482] T 00000000000000000000000000000000 P dddf2ff4a77a4e71a9473a1975529123 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:10:53.720957 17455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dddf2ff4a77a4e71a9473a1975529123 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:53.722072 17455 raft_consensus.cc:504] T 00000000000000000000000000000000 P dddf2ff4a77a4e71a9473a1975529123 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dddf2ff4a77a4e71a9473a1975529123" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42629 } }
I20260806 21:10:53.722450 17455 leader_election.cc:302] T 00000000000000000000000000000000 P dddf2ff4a77a4e71a9473a1975529123 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dddf2ff4a77a4e71a9473a1975529123; no voters:
I20260806 21:10:53.722815 17455 leader_election.cc:288] T 00000000000000000000000000000000 P dddf2ff4a77a4e71a9473a1975529123 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:10:53.723003 17465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dddf2ff4a77a4e71a9473a1975529123 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:10:53.723342 17465 raft_consensus.cc:686] T 00000000000000000000000000000000 P dddf2ff4a77a4e71a9473a1975529123 [term 1 LEADER]: Becoming Leader. State: Replica: dddf2ff4a77a4e71a9473a1975529123, State: Running, Role: LEADER
I20260806 21:10:53.723677 17465 consensus_queue.cc:237] T 00000000000000000000000000000000 P dddf2ff4a77a4e71a9473a1975529123 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dddf2ff4a77a4e71a9473a1975529123" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42629 } }
I20260806 21:10:53.724085 17455 sys_catalog.cc:564] T 00000000000000000000000000000000 P dddf2ff4a77a4e71a9473a1975529123 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:10:53.725440 17467 sys_catalog.cc:455] T 00000000000000000000000000000000 P dddf2ff4a77a4e71a9473a1975529123 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dddf2ff4a77a4e71a9473a1975529123. Latest consensus state: current_term: 1 leader_uuid: "dddf2ff4a77a4e71a9473a1975529123" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dddf2ff4a77a4e71a9473a1975529123" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42629 } } }
I20260806 21:10:53.725498 17466 sys_catalog.cc:455] T 00000000000000000000000000000000 P dddf2ff4a77a4e71a9473a1975529123 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dddf2ff4a77a4e71a9473a1975529123" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dddf2ff4a77a4e71a9473a1975529123" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42629 } } }
I20260806 21:10:53.725584 17467 sys_catalog.cc:458] T 00000000000000000000000000000000 P dddf2ff4a77a4e71a9473a1975529123 [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:53.725610 17466 sys_catalog.cc:458] T 00000000000000000000000000000000 P dddf2ff4a77a4e71a9473a1975529123 [sys.catalog]: This master's current role is: LEADER
W20260806 21:10:53.729467 17480 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dddf2ff4a77a4e71a9473a1975529123: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:10:53.729547 17480 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:10:53.729632 17475 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:10:53.730392 17475 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:10:53.735970 17475 catalog_manager.cc:1347] Generated new cluster ID: 9da3f504e8b54514b1ed01b8654530d7
I20260806 21:10:53.736056 17475 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:10:53.755700 17475 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:10:53.756060 17475 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:10:53.766829 17475 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dddf2ff4a77a4e71a9473a1975529123: Generated new TSK 0
I20260806 21:10:53.767714 17475 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:10:53.775501 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 17385
2026-08-06T21:10:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1456 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:10:53 dist-test-slave-69rg krb5kdc[17486](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:10:53 dist-test-slave-69rg krb5kdc[17486](info): set up 2 sockets
Aug 06 21:10:53 dist-test-slave-69rg krb5kdc[17486](info): commencing operation
krb5kdc: starting...
W20260806 21:10:54.861651 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:10:54 dist-test-slave-69rg krb5kdc[17486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:10:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:10:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 21:10:54.998893 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44177
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:10:55.128747 17499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:55.129037 17499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:55.129081 17499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:55.133126 17499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:10:55.133198 17499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:55.133219 17499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:55.133235 17499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:10:55.133257 17499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:10:55.138752 17499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44177
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:55.139811 17499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:55.140795 17499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:55.149730 17505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:55.149721 17508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:55.149756 17506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:55.150173 17499 server_base.cc:1034] running on GCE node
I20260806 21:10:55.150712 17499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:55.151810 17499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:55.153000 17499 hybrid_clock.cc:648] HybridClock initialized: now 1786050655152969 us; error 43 us; skew 500 ppm
Aug 06 21:10:55 dist-test-slave-69rg krb5kdc[17486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050655, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:55.155825 17499 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 21:10:55.156775 17499 webserver.cc:460] Webserver started at http://127.0.0.1:39283/ using document root <none> and password file <none>
I20260806 21:10:55.157351 17499 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:55.157442 17499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:55.157681 17499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:55.159816 17499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "ad1746671ed2483db3dbadc4a6dbd158"
format_stamp: "Formatted at 2026-08-06 21:10:55 on dist-test-slave-69rg"
server_key: "331d9794bbd3cc1cb808c3ffb8a52657"
server_key_iv: "01d44b048d8b99b47095ca8176b755db"
server_key_version: "encryptionkey@0"
I20260806 21:10:55.160419 17499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "ad1746671ed2483db3dbadc4a6dbd158"
format_stamp: "Formatted at 2026-08-06 21:10:55 on dist-test-slave-69rg"
server_key: "331d9794bbd3cc1cb808c3ffb8a52657"
server_key_iv: "01d44b048d8b99b47095ca8176b755db"
server_key_version: "encryptionkey@0"
I20260806 21:10:55.164404 17499 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 21:10:55.166908 17516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:55.168224 17499 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:10:55.168396 17499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "ad1746671ed2483db3dbadc4a6dbd158"
format_stamp: "Formatted at 2026-08-06 21:10:55 on dist-test-slave-69rg"
server_key: "331d9794bbd3cc1cb808c3ffb8a52657"
server_key_iv: "01d44b048d8b99b47095ca8176b755db"
server_key_version: "encryptionkey@0"
I20260806 21:10:55.168515 17499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:55.191129 17499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:55.194252 17499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:55.194499 17499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:55.202025 17499 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44177
I20260806 21:10:55.202092 17569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44177 every 8 connection(s)
I20260806 21:10:55.203022 17499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:10:55.205734 17570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:55.206825 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 17499
I20260806 21:10:55.206970 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:10:55.207319 28551 external_mini_cluster.cc:1428] Setting key 1937bdbe91f9e6369222e9d5928f0c7d
I20260806 21:10:55.212098 17570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad1746671ed2483db3dbadc4a6dbd158: Bootstrap starting.
I20260806 21:10:55.214412 17570 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad1746671ed2483db3dbadc4a6dbd158: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:55.215189 17570 log.cc:824] T 00000000000000000000000000000000 P ad1746671ed2483db3dbadc4a6dbd158: Log is configured to *not* fsync() on all Append() calls
Aug 06 21:10:55 dist-test-slave-69rg krb5kdc[17486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 21:10:55.217520 17570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad1746671ed2483db3dbadc4a6dbd158: No bootstrap required, opened a new log
I20260806 21:10:55.221702 17573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:55.208804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:55695 (local address 127.0.0.1:44177)
0806 21:10:55.209257 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:55.209280 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:55.209318 (+ 38us) server_negotiation.cc:407] Connection header received
0806 21:10:55.210031 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:55.210050 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:55.210394 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:55.210720 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:55.211523 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:55.212533 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:55.213386 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:55.213915 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:55.214072 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:55.216907 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:55.216912 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:55.216925 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:10:55.216950 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:55.218705 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:55.219297 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:55.219302 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:55.219306 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:55.219413 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:55.219817 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:55.219819 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:55.219820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:55.220201 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:55.220361 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:55.220689 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 21:10:55.220939 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":139,"threads_started":1}
I20260806 21:10:55.221891 17570 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad1746671ed2483db3dbadc4a6dbd158 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1746671ed2483db3dbadc4a6dbd158" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44177 } }
I20260806 21:10:55.222177 17570 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad1746671ed2483db3dbadc4a6dbd158 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:55.222252 17570 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad1746671ed2483db3dbadc4a6dbd158 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad1746671ed2483db3dbadc4a6dbd158, State: Initialized, Role: FOLLOWER
I20260806 21:10:55.222695 17570 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad1746671ed2483db3dbadc4a6dbd158 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1746671ed2483db3dbadc4a6dbd158" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44177 } }
I20260806 21:10:55.222843 17570 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad1746671ed2483db3dbadc4a6dbd158 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:10:55.222916 17570 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad1746671ed2483db3dbadc4a6dbd158 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:10:55.223032 17570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad1746671ed2483db3dbadc4a6dbd158 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:55.224066 17570 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad1746671ed2483db3dbadc4a6dbd158 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1746671ed2483db3dbadc4a6dbd158" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44177 } }
I20260806 21:10:55.224409 17570 leader_election.cc:302] T 00000000000000000000000000000000 P ad1746671ed2483db3dbadc4a6dbd158 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad1746671ed2483db3dbadc4a6dbd158; no voters:
I20260806 21:10:55.224745 17570 leader_election.cc:288] T 00000000000000000000000000000000 P ad1746671ed2483db3dbadc4a6dbd158 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:10:55.224836 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:10:55.224848 17575 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad1746671ed2483db3dbadc4a6dbd158 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:10:55.225054 17575 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad1746671ed2483db3dbadc4a6dbd158 [term 1 LEADER]: Becoming Leader. State: Replica: ad1746671ed2483db3dbadc4a6dbd158, State: Running, Role: LEADER
I20260806 21:10:55.225404 17575 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad1746671ed2483db3dbadc4a6dbd158 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1746671ed2483db3dbadc4a6dbd158" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44177 } }
I20260806 21:10:55.226127 17570 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad1746671ed2483db3dbadc4a6dbd158 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:10:55.226635 17575 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1746671ed2483db3dbadc4a6dbd158 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad1746671ed2483db3dbadc4a6dbd158" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1746671ed2483db3dbadc4a6dbd158" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44177 } } }
I20260806 21:10:55.226838 17575 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1746671ed2483db3dbadc4a6dbd158 [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:55.227061 17576 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1746671ed2483db3dbadc4a6dbd158 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad1746671ed2483db3dbadc4a6dbd158. Latest consensus state: current_term: 1 leader_uuid: "ad1746671ed2483db3dbadc4a6dbd158" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1746671ed2483db3dbadc4a6dbd158" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44177 } } }
I20260806 21:10:55.227169 17576 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1746671ed2483db3dbadc4a6dbd158 [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:55.227308 17585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:10:55.230432 17585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:10:55.236976 17585 catalog_manager.cc:1347] Generated new cluster ID: 1311a9d7bdb44b62a0ebee28236d8e70
I20260806 21:10:55.237046 17585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:10:55.238817 17573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:55.228508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:41431 (local address 127.0.0.1:44177)
0806 21:10:55.228616 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:55.228621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:55.230110 (+ 1489us) server_negotiation.cc:407] Connection header received
0806 21:10:55.230247 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:55.230252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:55.230316 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:55.230407 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:55.231274 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:55.232076 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:55.233032 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:55.233287 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:55.233392 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:55.234981 (+ 1589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:55.234986 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:55.234989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:10:55.235016 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:55.236804 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:55.237412 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:55.237416 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:55.237417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:55.237511 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:55.237913 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:55.237916 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:55.237917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:55.238129 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:55.238281 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:55.238574 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 21:10:55.238670 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 21:10:55.250473 17585 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:10:55.251704 17585 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:10:55.257965 17585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad1746671ed2483db3dbadc4a6dbd158: Generated new TSK 0
I20260806 21:10:55.258683 17585 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:10:55.264397 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 17499
W20260806 21:10:55.271220 17484 connection.cc:535] client connection to 127.0.0.1:44177 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44177): Connection reset by peer (error 104)
2026-08-06T21:10:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1488 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:10:55 dist-test-slave-69rg krb5kdc[17601](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:10:55 dist-test-slave-69rg krb5kdc[17601](info): set up 2 sockets
Aug 06 21:10:55 dist-test-slave-69rg krb5kdc[17601](info): commencing operation
krb5kdc: starting...
W20260806 21:10:56.377574 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:10:56 dist-test-slave-69rg krb5kdc[17601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:10:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:10:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 21:10:56.513036 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34675
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:10:56.631111 17614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:56.631394 17614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:56.631436 17614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:56.635572 17614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:10:56.635658 17614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:56.635679 17614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:56.635695 17614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:10:56.635711 17614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:10:56.641264 17614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:56.642436 17614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:56.643430 17614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:56.652453 17620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:56.652534 17619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:56.652631 17622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:56.652750 17614 server_base.cc:1034] running on GCE node
I20260806 21:10:56.653439 17614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:56.654672 17614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:56.655848 17614 hybrid_clock.cc:648] HybridClock initialized: now 1786050656655830 us; error 31 us; skew 500 ppm
Aug 06 21:10:56 dist-test-slave-69rg krb5kdc[17601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050656, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:56.658864 17614 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 21:10:56.659863 17614 webserver.cc:460] Webserver started at http://127.0.0.1:33725/ using document root <none> and password file <none>
I20260806 21:10:56.660455 17614 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:56.660517 17614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:56.660729 17614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:56.662870 17614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "0e73de8f87c041d2bec3861563c85796"
format_stamp: "Formatted at 2026-08-06 21:10:56 on dist-test-slave-69rg"
server_key: "87e1fd0fd693e338de6d3f9a77d34cfa"
server_key_iv: "b7202384ca1f77a8811120ff80922a49"
server_key_version: "encryptionkey@0"
I20260806 21:10:56.663460 17614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "0e73de8f87c041d2bec3861563c85796"
format_stamp: "Formatted at 2026-08-06 21:10:56 on dist-test-slave-69rg"
server_key: "87e1fd0fd693e338de6d3f9a77d34cfa"
server_key_iv: "b7202384ca1f77a8811120ff80922a49"
server_key_version: "encryptionkey@0"
I20260806 21:10:56.667408 17614 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:10:56.669965 17629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:56.671293 17614 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:10:56.671434 17614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "0e73de8f87c041d2bec3861563c85796"
format_stamp: "Formatted at 2026-08-06 21:10:56 on dist-test-slave-69rg"
server_key: "87e1fd0fd693e338de6d3f9a77d34cfa"
server_key_iv: "b7202384ca1f77a8811120ff80922a49"
server_key_version: "encryptionkey@0"
I20260806 21:10:56.671550 17614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:56.687690 17614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:56.690850 17614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:56.691064 17614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:56.698254 17614 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34675
I20260806 21:10:56.698285 17682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34675 every 8 connection(s)
I20260806 21:10:56.699239 17614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:10:56.700678 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 17614
I20260806 21:10:56.700824 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:10:56.701185 28551 external_mini_cluster.cc:1428] Setting key adcbd725fcb9c912f44715b05df966d0
I20260806 21:10:56.702339 17683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:56.708297 17683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e73de8f87c041d2bec3861563c85796: Bootstrap starting.
I20260806 21:10:56.710414 17686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:56.702685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:45193 (local address 127.0.0.1:34675)
0806 21:10:56.703213 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:56.703228 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:56.703266 (+ 38us) server_negotiation.cc:407] Connection header received
0806 21:10:56.703936 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:56.703952 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:56.704298 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:56.704656 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:56.705074 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:56.706157 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:56.707056 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:56.707534 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:56.707702 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:56.708675 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:56.708679 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:56.708693 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:56.709086 (+ 393us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:56.709142 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:56.709164 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:56.709304 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:56.709490 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 21:10:56.709628 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":188,"threads_started":1}
I20260806 21:10:56.710850 17683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e73de8f87c041d2bec3861563c85796: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:56.711716 17683 log.cc:824] T 00000000000000000000000000000000 P 0e73de8f87c041d2bec3861563c85796: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:56.713586 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:10:56.714107 17683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e73de8f87c041d2bec3861563c85796: No bootstrap required, opened a new log
I20260806 21:10:56.718974 17683 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e73de8f87c041d2bec3861563c85796 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e73de8f87c041d2bec3861563c85796" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34675 } }
I20260806 21:10:56.719333 17683 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e73de8f87c041d2bec3861563c85796 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:56.719401 17683 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e73de8f87c041d2bec3861563c85796 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e73de8f87c041d2bec3861563c85796, State: Initialized, Role: FOLLOWER
I20260806 21:10:56.719911 17683 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e73de8f87c041d2bec3861563c85796 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e73de8f87c041d2bec3861563c85796" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34675 } }
I20260806 21:10:56.720098 17683 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e73de8f87c041d2bec3861563c85796 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:10:56.720198 17683 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e73de8f87c041d2bec3861563c85796 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:10:56.720309 17683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e73de8f87c041d2bec3861563c85796 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:56.721004 17686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:56.715992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:52937 (local address 127.0.0.1:34675)
0806 21:10:56.716121 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:56.716128 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:56.716341 (+ 213us) server_negotiation.cc:407] Connection header received
0806 21:10:56.716427 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:56.716430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:56.716489 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:56.716597 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:56.716911 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:56.717713 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:56.718688 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:56.719015 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:56.719130 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:56.720431 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:56.720435 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:56.720438 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:56.720453 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:56.720490 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:56.720502 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:56.720579 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:56.720785 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 21:10:56.720850 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 21:10:56.721680 17683 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e73de8f87c041d2bec3861563c85796 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e73de8f87c041d2bec3861563c85796" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34675 } }
I20260806 21:10:56.722112 17683 leader_election.cc:302] T 00000000000000000000000000000000 P 0e73de8f87c041d2bec3861563c85796 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e73de8f87c041d2bec3861563c85796; no voters:
I20260806 21:10:56.722496 17683 leader_election.cc:288] T 00000000000000000000000000000000 P 0e73de8f87c041d2bec3861563c85796 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:10:56.722659 17693 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e73de8f87c041d2bec3861563c85796 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:10:56.722905 17693 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e73de8f87c041d2bec3861563c85796 [term 1 LEADER]: Becoming Leader. State: Replica: 0e73de8f87c041d2bec3861563c85796, State: Running, Role: LEADER
I20260806 21:10:56.723192 17693 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e73de8f87c041d2bec3861563c85796 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e73de8f87c041d2bec3861563c85796" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34675 } }
I20260806 21:10:56.723793 17683 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e73de8f87c041d2bec3861563c85796 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:10:56.725260 17695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e73de8f87c041d2bec3861563c85796 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e73de8f87c041d2bec3861563c85796. Latest consensus state: current_term: 1 leader_uuid: "0e73de8f87c041d2bec3861563c85796" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e73de8f87c041d2bec3861563c85796" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34675 } } }
I20260806 21:10:56.725402 17695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e73de8f87c041d2bec3861563c85796 [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:56.725265 17694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e73de8f87c041d2bec3861563c85796 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e73de8f87c041d2bec3861563c85796" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e73de8f87c041d2bec3861563c85796" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34675 } } }
I20260806 21:10:56.725797 17694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e73de8f87c041d2bec3861563c85796 [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:56.726177 17702 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:10:56.729652 17702 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:10:56.735394 17702 catalog_manager.cc:1347] Generated new cluster ID: 1fe95dde200441c9b379d77e9f0e697b
I20260806 21:10:56.735505 17702 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:10:56.762951 17702 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:10:56.764302 17702 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:10:56.774617 17702 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e73de8f87c041d2bec3861563c85796: Generated new TSK 0
I20260806 21:10:56.775596 17702 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:10:56.782341 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 17614
W20260806 21:10:56.789634 17599 connection.cc:535] client connection to 127.0.0.1:34675 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34675): Connection reset by peer (error 104)
2026-08-06T21:10:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1518 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:10:56 dist-test-slave-69rg krb5kdc[17714](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:10:56 dist-test-slave-69rg krb5kdc[17714](info): set up 2 sockets
Aug 06 21:10:56 dist-test-slave-69rg krb5kdc[17714](info): commencing operation
krb5kdc: starting...
W20260806 21:10:57.833585 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:10:57 dist-test-slave-69rg krb5kdc[17714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:10:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:10:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 21:10:57.973935 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35939
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35939
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:10:58.093686 17727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:58.094090 17727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:58.094180 17727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:58.098486 17727 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:10:58.098615 17727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:58.098652 17727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:58.098690 17727 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:10:58.098730 17727 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:10:58.104882 17727 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:58.106148 17727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:58.107247 17727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:58.116353 17733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:58.116351 17735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:58.116344 17732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:58.116508 17727 server_base.cc:1034] running on GCE node
I20260806 21:10:58.117154 17727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:58.118326 17727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:58.119517 17727 hybrid_clock.cc:648] HybridClock initialized: now 1786050658119494 us; error 36 us; skew 500 ppm
Aug 06 21:10:58 dist-test-slave-69rg krb5kdc[17714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050658, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:58.122428 17727 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 21:10:58.123369 17727 webserver.cc:460] Webserver started at http://127.0.0.1:33441/ using document root <none> and password file <none>
I20260806 21:10:58.123948 17727 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:58.124040 17727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:58.124274 17727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:58.126562 17727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "0fe876d6f3d345668f36bc5bcf439acc"
format_stamp: "Formatted at 2026-08-06 21:10:58 on dist-test-slave-69rg"
server_key: "fde98ca915cf49e8395fc93196186127"
server_key_iv: "e0ffe71050d8e906963310ff5c90baed"
server_key_version: "encryptionkey@0"
I20260806 21:10:58.127161 17727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "0fe876d6f3d345668f36bc5bcf439acc"
format_stamp: "Formatted at 2026-08-06 21:10:58 on dist-test-slave-69rg"
server_key: "fde98ca915cf49e8395fc93196186127"
server_key_iv: "e0ffe71050d8e906963310ff5c90baed"
server_key_version: "encryptionkey@0"
I20260806 21:10:58.131062 17727 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260806 21:10:58.133616 17743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:58.134956 17727 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:10:58.135111 17727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "0fe876d6f3d345668f36bc5bcf439acc"
format_stamp: "Formatted at 2026-08-06 21:10:58 on dist-test-slave-69rg"
server_key: "fde98ca915cf49e8395fc93196186127"
server_key_iv: "e0ffe71050d8e906963310ff5c90baed"
server_key_version: "encryptionkey@0"
I20260806 21:10:58.135236 17727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:58.140086 17727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:58.143257 17727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:58.143461 17727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:58.150795 17727 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35939
I20260806 21:10:58.150849 17795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35939 every 8 connection(s)
I20260806 21:10:58.151849 17727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:10:58.154788 17796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:58.160404 17796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fe876d6f3d345668f36bc5bcf439acc: Bootstrap starting.
I20260806 21:10:58.162009 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 17727
I20260806 21:10:58.162163 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:10:58.162467 28551 external_mini_cluster.cc:1428] Setting key d7c3a6833fe563c21375e31bbc324b0d
I20260806 21:10:58.163192 17796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fe876d6f3d345668f36bc5bcf439acc: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:58.164588 17796 log.cc:824] T 00000000000000000000000000000000 P 0fe876d6f3d345668f36bc5bcf439acc: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:58.166671 17796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fe876d6f3d345668f36bc5bcf439acc: No bootstrap required, opened a new log
I20260806 21:10:58.167640 17799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:58.163645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:51517 (local address 127.0.0.1:35939)
0806 21:10:58.164168 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:58.164184 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:58.164220 (+ 36us) server_negotiation.cc:407] Connection header received
0806 21:10:58.164968 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:58.164988 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:58.165270 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:58.165610 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:58.166436 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:58.166440 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:58.166451 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:58.166462 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:58.166515 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:58.166534 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:58.166649 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:58.166756 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 21:10:58.166926 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":151,"threads_started":1}
I20260806 21:10:58.170369 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:10:58.170568 17796 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fe876d6f3d345668f36bc5bcf439acc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe876d6f3d345668f36bc5bcf439acc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35939 } }
I20260806 21:10:58.170841 17796 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fe876d6f3d345668f36bc5bcf439acc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:58.170912 17796 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fe876d6f3d345668f36bc5bcf439acc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fe876d6f3d345668f36bc5bcf439acc, State: Initialized, Role: FOLLOWER
I20260806 21:10:58.171398 17796 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fe876d6f3d345668f36bc5bcf439acc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe876d6f3d345668f36bc5bcf439acc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35939 } }
I20260806 21:10:58.171561 17796 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fe876d6f3d345668f36bc5bcf439acc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:10:58.171633 17796 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fe876d6f3d345668f36bc5bcf439acc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:10:58.171763 17796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fe876d6f3d345668f36bc5bcf439acc [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:58.173072 17796 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fe876d6f3d345668f36bc5bcf439acc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe876d6f3d345668f36bc5bcf439acc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35939 } }
I20260806 21:10:58.173494 17796 leader_election.cc:302] T 00000000000000000000000000000000 P 0fe876d6f3d345668f36bc5bcf439acc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fe876d6f3d345668f36bc5bcf439acc; no voters:
I20260806 21:10:58.173920 17796 leader_election.cc:288] T 00000000000000000000000000000000 P 0fe876d6f3d345668f36bc5bcf439acc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:10:58.174074 17806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fe876d6f3d345668f36bc5bcf439acc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:10:58.174336 17806 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fe876d6f3d345668f36bc5bcf439acc [term 1 LEADER]: Becoming Leader. State: Replica: 0fe876d6f3d345668f36bc5bcf439acc, State: Running, Role: LEADER
I20260806 21:10:58.174695 17806 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fe876d6f3d345668f36bc5bcf439acc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe876d6f3d345668f36bc5bcf439acc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35939 } }
I20260806 21:10:58.174928 17799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:58.172882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:44963 (local address 127.0.0.1:35939)
0806 21:10:58.173025 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:58.173030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:58.173204 (+ 174us) server_negotiation.cc:407] Connection header received
0806 21:10:58.173329 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:58.173331 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:58.173364 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:58.173432 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:58.174360 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:58.174364 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:58.174367 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:10:58.174380 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:58.174412 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:10:58.174426 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:58.174527 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:58.174763 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 21:10:58.174805 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 21:10:58.175410 17796 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fe876d6f3d345668f36bc5bcf439acc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:10:58.176057 17806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fe876d6f3d345668f36bc5bcf439acc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fe876d6f3d345668f36bc5bcf439acc. Latest consensus state: current_term: 1 leader_uuid: "0fe876d6f3d345668f36bc5bcf439acc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe876d6f3d345668f36bc5bcf439acc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35939 } } }
I20260806 21:10:58.176190 17806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fe876d6f3d345668f36bc5bcf439acc [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:58.176486 17807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fe876d6f3d345668f36bc5bcf439acc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fe876d6f3d345668f36bc5bcf439acc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe876d6f3d345668f36bc5bcf439acc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35939 } } }
I20260806 21:10:58.176584 17807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fe876d6f3d345668f36bc5bcf439acc [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:58.176638 17814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:10:58.180032 17814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:10:58.186153 17814 catalog_manager.cc:1347] Generated new cluster ID: 67ec90bebe5b478cade2d595aad028e7
I20260806 21:10:58.186264 17814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:10:58.200726 17814 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:10:58.201059 17814 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:10:58.208925 17814 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fe876d6f3d345668f36bc5bcf439acc: Generated new TSK 0
I20260806 21:10:58.209856 17814 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:10:58.220363 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 17727
2026-08-06T21:10:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1437 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:10:58 dist-test-slave-69rg krb5kdc[17827](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:10:58 dist-test-slave-69rg krb5kdc[17827](info): set up 2 sockets
Aug 06 21:10:58 dist-test-slave-69rg krb5kdc[17827](info): commencing operation
krb5kdc: starting...
W20260806 21:10:59.282899 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:10:59 dist-test-slave-69rg krb5kdc[17827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:10:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:10:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 21:10:59.422097 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33795
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:10:59.536585 17840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:10:59.536902 17840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:10:59.536978 17840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:10:59.540982 17840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:10:59.541082 17840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:10:59.541118 17840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:10:59.541157 17840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:10:59.541189 17840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:10:59.546470 17840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33795
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:10:59.547618 17840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:10:59.548651 17840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:10:59.557317 17845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:59.557534 17840 server_base.cc:1034] running on GCE node
W20260806 21:10:59.557308 17846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:10:59.557385 17848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:10:59.558248 17840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:10:59.559353 17840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:10:59.560537 17840 hybrid_clock.cc:648] HybridClock initialized: now 1786050659560527 us; error 28 us; skew 500 ppm
Aug 06 21:10:59 dist-test-slave-69rg krb5kdc[17827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050659, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:10:59.567750 17840 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 21:10:59.568665 17840 webserver.cc:460] Webserver started at http://127.0.0.1:33515/ using document root <none> and password file <none>
I20260806 21:10:59.569193 17840 fs_manager.cc:359] Metadata directory not provided
I20260806 21:10:59.569257 17840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:10:59.569422 17840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:10:59.571524 17840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "3964d96485b44a21a1df90504ef4455c"
format_stamp: "Formatted at 2026-08-06 21:10:59 on dist-test-slave-69rg"
server_key: "5d6f8aaa635c3ce7a751d36558e7f39a"
server_key_iv: "c9ef7219ca59e1838e77441bd48280df"
server_key_version: "encryptionkey@0"
I20260806 21:10:59.572073 17840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "3964d96485b44a21a1df90504ef4455c"
format_stamp: "Formatted at 2026-08-06 21:10:59 on dist-test-slave-69rg"
server_key: "5d6f8aaa635c3ce7a751d36558e7f39a"
server_key_iv: "c9ef7219ca59e1838e77441bd48280df"
server_key_version: "encryptionkey@0"
I20260806 21:10:59.575865 17840 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:10:59.578294 17855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:10:59.579519 17840 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:10:59.579643 17840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "3964d96485b44a21a1df90504ef4455c"
format_stamp: "Formatted at 2026-08-06 21:10:59 on dist-test-slave-69rg"
server_key: "5d6f8aaa635c3ce7a751d36558e7f39a"
server_key_iv: "c9ef7219ca59e1838e77441bd48280df"
server_key_version: "encryptionkey@0"
I20260806 21:10:59.579766 17840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:10:59.600222 17840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:10:59.603521 17840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:10:59.603739 17840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:10:59.610958 17840 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33795
I20260806 21:10:59.610986 17908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33795 every 8 connection(s)
I20260806 21:10:59.611929 17840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:10:59.614694 17909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:10:59.620189 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 17840
I20260806 21:10:59.620338 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:10:59.620319 17909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3964d96485b44a21a1df90504ef4455c: Bootstrap starting.
I20260806 21:10:59.620645 28551 external_mini_cluster.cc:1428] Setting key 7745a080497616cd8d7bf94f72cdd9b0
I20260806 21:10:59.622975 17909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3964d96485b44a21a1df90504ef4455c: Neither blocks nor log segments found. Creating new log.
I20260806 21:10:59.623706 17909 log.cc:824] T 00000000000000000000000000000000 P 3964d96485b44a21a1df90504ef4455c: Log is configured to *not* fsync() on all Append() calls
I20260806 21:10:59.625531 17909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3964d96485b44a21a1df90504ef4455c: No bootstrap required, opened a new log
I20260806 21:10:59.628960 17909 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3964d96485b44a21a1df90504ef4455c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3964d96485b44a21a1df90504ef4455c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33795 } }
I20260806 21:10:59.629184 17909 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3964d96485b44a21a1df90504ef4455c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:10:59.629261 17909 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3964d96485b44a21a1df90504ef4455c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3964d96485b44a21a1df90504ef4455c, State: Initialized, Role: FOLLOWER
Aug 06 21:10:59 dist-test-slave-69rg krb5kdc[17827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 21:10:59.629694 17909 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3964d96485b44a21a1df90504ef4455c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3964d96485b44a21a1df90504ef4455c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33795 } }
I20260806 21:10:59.629863 17909 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3964d96485b44a21a1df90504ef4455c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:10:59.629940 17909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3964d96485b44a21a1df90504ef4455c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:10:59.630064 17909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3964d96485b44a21a1df90504ef4455c [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:10:59.630940 17909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3964d96485b44a21a1df90504ef4455c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3964d96485b44a21a1df90504ef4455c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33795 } }
I20260806 21:10:59.631273 17909 leader_election.cc:302] T 00000000000000000000000000000000 P 3964d96485b44a21a1df90504ef4455c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3964d96485b44a21a1df90504ef4455c; no voters:
I20260806 21:10:59.631562 17909 leader_election.cc:288] T 00000000000000000000000000000000 P 3964d96485b44a21a1df90504ef4455c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:10:59.631664 17914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3964d96485b44a21a1df90504ef4455c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:10:59.631912 17914 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3964d96485b44a21a1df90504ef4455c [term 1 LEADER]: Becoming Leader. State: Replica: 3964d96485b44a21a1df90504ef4455c, State: Running, Role: LEADER
I20260806 21:10:59.632766 17909 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3964d96485b44a21a1df90504ef4455c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:10:59.632766 17914 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3964d96485b44a21a1df90504ef4455c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3964d96485b44a21a1df90504ef4455c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33795 } }
I20260806 21:10:59.634606 17916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3964d96485b44a21a1df90504ef4455c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3964d96485b44a21a1df90504ef4455c. Latest consensus state: current_term: 1 leader_uuid: "3964d96485b44a21a1df90504ef4455c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3964d96485b44a21a1df90504ef4455c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33795 } } }
I20260806 21:10:59.634757 17916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3964d96485b44a21a1df90504ef4455c [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:59.634606 17915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3964d96485b44a21a1df90504ef4455c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3964d96485b44a21a1df90504ef4455c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3964d96485b44a21a1df90504ef4455c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33795 } } }
I20260806 21:10:59.634881 17915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3964d96485b44a21a1df90504ef4455c [sys.catalog]: This master's current role is: LEADER
I20260806 21:10:59.636058 17912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:59.622114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:40397 (local address 127.0.0.1:33795)
0806 21:10:59.622545 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:59.622564 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:59.622598 (+ 34us) server_negotiation.cc:407] Connection header received
0806 21:10:59.623340 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:59.623358 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:59.623692 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:59.624050 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:59.624860 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:59.625927 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:59.626768 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:59.627233 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:59.627391 (+ 158us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:59.629982 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:59.629988 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:59.630007 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:10:59.630052 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:59.632098 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:59.632654 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:59.632659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:59.632665 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:59.632809 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:59.633220 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:59.633224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:59.633225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:59.633796 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:59.634052 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:59.634861 (+ 809us) server_negotiation.cc:299] Negotiation successful
0806 21:10:59.635100 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":138,"threads_started":1}
W20260806 21:10:59.637754 17929 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3964d96485b44a21a1df90504ef4455c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:10:59.637859 17929 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:10:59.637976 17928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:10:59.638945 17928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:10:59.639494 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:10:59.645190 17928 catalog_manager.cc:1347] Generated new cluster ID: da0dcd4318a748738aa27dfce33c361e
I20260806 21:10:59.645267 17928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:10:59.651060 17912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:10:59.641913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:41719 (local address 127.0.0.1:33795)
0806 21:10:59.642029 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 21:10:59.642034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:10:59.642315 (+ 281us) server_negotiation.cc:407] Connection header received
0806 21:10:59.642444 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:10:59.642447 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:10:59.642500 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:10:59.642587 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:10:59.643489 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:59.644030 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:59.644964 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:10:59.645337 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:10:59.645438 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:10:59.647308 (+ 1870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:10:59.647312 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:10:59.647315 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:10:59.647343 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:10:59.649084 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:59.649620 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:59.649624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:59.649626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:59.649722 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:10:59.650094 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:10:59.650097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:10:59.650099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:10:59.650342 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:10:59.650510 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 21:10:59.650782 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 21:10:59.650892 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 21:10:59.662595 17928 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:10:59.663869 17928 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:10:59.679611 17928 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3964d96485b44a21a1df90504ef4455c: Generated new TSK 0
I20260806 21:10:59.680565 17928 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:10:59.689054 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 17840
W20260806 21:10:59.696094 17825 connection.cc:535] client connection to 127.0.0.1:33795 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33795): Connection reset by peer (error 104)
2026-08-06T21:10:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1468 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:10:59 dist-test-slave-69rg krb5kdc[17940](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:10:59 dist-test-slave-69rg krb5kdc[17940](info): set up 2 sockets
Aug 06 21:10:59 dist-test-slave-69rg krb5kdc[17940](info): commencing operation
krb5kdc: starting...
W20260806 21:11:00.773594 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:11:00 dist-test-slave-69rg krb5kdc[17940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:11:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:11:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 21:11:00.913609 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43039
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:01.047562 17953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:01.048053 17953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:01.048161 17953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:01.052976 17953 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:01.053141 17953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:01.053184 17953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:01.053236 17953 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:01.053289 17953 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:11:01.059651 17953 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43039
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:01.060930 17953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:01.062112 17953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:11:01.072546 17953 server_base.cc:1034] running on GCE node
W20260806 21:11:01.072551 17959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:01.072795 17962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:01.072551 17960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:01.073729 17953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:01.075066 17953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:01.076327 17953 hybrid_clock.cc:648] HybridClock initialized: now 1786050661076310 us; error 34 us; skew 500 ppm
Aug 06 21:11:01 dist-test-slave-69rg krb5kdc[17940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050661, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:01.080125 17953 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 21:11:01.081295 17953 webserver.cc:460] Webserver started at http://127.0.0.1:35851/ using document root <none> and password file <none>
I20260806 21:11:01.081970 17953 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:01.082070 17953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:01.082330 17953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:01.085454 17953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "40c8e0f171e248a7a5cb8c00d73ceafb"
format_stamp: "Formatted at 2026-08-06 21:11:01 on dist-test-slave-69rg"
server_key: "6dff9ffdb58d4a3cd1c0a62108f39f46"
server_key_iv: "4f3f8e9538873e6025cda140bdb45f53"
server_key_version: "encryptionkey@0"
I20260806 21:11:01.086462 17953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "40c8e0f171e248a7a5cb8c00d73ceafb"
format_stamp: "Formatted at 2026-08-06 21:11:01 on dist-test-slave-69rg"
server_key: "6dff9ffdb58d4a3cd1c0a62108f39f46"
server_key_iv: "4f3f8e9538873e6025cda140bdb45f53"
server_key_version: "encryptionkey@0"
I20260806 21:11:01.091442 17953 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260806 21:11:01.094617 17969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:01.096601 17953 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:11:01.096794 17953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "40c8e0f171e248a7a5cb8c00d73ceafb"
format_stamp: "Formatted at 2026-08-06 21:11:01 on dist-test-slave-69rg"
server_key: "6dff9ffdb58d4a3cd1c0a62108f39f46"
server_key_iv: "4f3f8e9538873e6025cda140bdb45f53"
server_key_version: "encryptionkey@0"
I20260806 21:11:01.096948 17953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:01.116287 17953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:01.119979 17953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:01.120239 17953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:01.129506 17953 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43039
I20260806 21:11:01.129537 18023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43039 every 8 connection(s)
I20260806 21:11:01.130625 17953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:11:01.133002 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 17953
I20260806 21:11:01.133181 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:11:01.133530 28551 external_mini_cluster.cc:1428] Setting key 47d5b5d79fa76016fbea8c0b22d9b56c
I20260806 21:11:01.133623 18024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:01.141417 18024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40c8e0f171e248a7a5cb8c00d73ceafb: Bootstrap starting.
I20260806 21:11:01.143735 18027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:01.135071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:35621 (local address 127.0.0.1:43039)
0806 21:11:01.135572 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:01.135588 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:01.135626 (+ 38us) server_negotiation.cc:407] Connection header received
0806 21:11:01.136303 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:01.136319 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:01.136656 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:01.136946 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:01.137388 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:01.138511 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:01.139902 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:01.140400 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:01.140531 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:01.141743 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:01.141749 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:01.141763 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:01.142239 (+ 476us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:01.142304 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:01.142330 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:01.142451 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:01.142721 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 21:11:01.142899 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":180,"threads_started":1}
I20260806 21:11:01.144495 18024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40c8e0f171e248a7a5cb8c00d73ceafb: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:01.145541 18024 log.cc:824] T 00000000000000000000000000000000 P 40c8e0f171e248a7a5cb8c00d73ceafb: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:01.147564 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:11:01.148331 18024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40c8e0f171e248a7a5cb8c00d73ceafb: No bootstrap required, opened a new log
I20260806 21:11:01.152709 18024 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40c8e0f171e248a7a5cb8c00d73ceafb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c8e0f171e248a7a5cb8c00d73ceafb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43039 } }
I20260806 21:11:01.153044 18024 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40c8e0f171e248a7a5cb8c00d73ceafb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:01.153100 18024 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40c8e0f171e248a7a5cb8c00d73ceafb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40c8e0f171e248a7a5cb8c00d73ceafb, State: Initialized, Role: FOLLOWER
I20260806 21:11:01.153795 18024 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40c8e0f171e248a7a5cb8c00d73ceafb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c8e0f171e248a7a5cb8c00d73ceafb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43039 } }
I20260806 21:11:01.154117 18024 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40c8e0f171e248a7a5cb8c00d73ceafb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:11:01.154240 18024 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40c8e0f171e248a7a5cb8c00d73ceafb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:11:01.154381 18024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40c8e0f171e248a7a5cb8c00d73ceafb [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:01.155170 18027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:01.150057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:59129 (local address 127.0.0.1:43039)
0806 21:11:01.150185 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:01.150190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:01.150366 (+ 176us) server_negotiation.cc:407] Connection header received
0806 21:11:01.150461 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:01.150464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:01.150515 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:01.150602 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:01.151123 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:01.151906 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:01.152960 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:01.153305 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:01.153420 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:01.154574 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:01.154581 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:01.154586 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:01.154600 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:01.154633 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:01.154651 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:01.154766 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:01.154986 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 21:11:01.155030 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 21:11:01.155905 18024 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40c8e0f171e248a7a5cb8c00d73ceafb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c8e0f171e248a7a5cb8c00d73ceafb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43039 } }
I20260806 21:11:01.156304 18024 leader_election.cc:302] T 00000000000000000000000000000000 P 40c8e0f171e248a7a5cb8c00d73ceafb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40c8e0f171e248a7a5cb8c00d73ceafb; no voters:
I20260806 21:11:01.156669 18024 leader_election.cc:288] T 00000000000000000000000000000000 P 40c8e0f171e248a7a5cb8c00d73ceafb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:11:01.156991 18034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40c8e0f171e248a7a5cb8c00d73ceafb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:01.157226 18034 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40c8e0f171e248a7a5cb8c00d73ceafb [term 1 LEADER]: Becoming Leader. State: Replica: 40c8e0f171e248a7a5cb8c00d73ceafb, State: Running, Role: LEADER
I20260806 21:11:01.157542 18034 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40c8e0f171e248a7a5cb8c00d73ceafb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c8e0f171e248a7a5cb8c00d73ceafb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43039 } }
I20260806 21:11:01.158020 18024 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40c8e0f171e248a7a5cb8c00d73ceafb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:01.159385 18035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40c8e0f171e248a7a5cb8c00d73ceafb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40c8e0f171e248a7a5cb8c00d73ceafb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c8e0f171e248a7a5cb8c00d73ceafb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43039 } } }
I20260806 21:11:01.159559 18035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40c8e0f171e248a7a5cb8c00d73ceafb [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:01.159960 18036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40c8e0f171e248a7a5cb8c00d73ceafb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40c8e0f171e248a7a5cb8c00d73ceafb. Latest consensus state: current_term: 1 leader_uuid: "40c8e0f171e248a7a5cb8c00d73ceafb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c8e0f171e248a7a5cb8c00d73ceafb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43039 } } }
I20260806 21:11:01.160074 18036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40c8e0f171e248a7a5cb8c00d73ceafb [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:01.160702 18046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:11:01.164579 18046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:11:01.171240 18046 catalog_manager.cc:1347] Generated new cluster ID: 86c8e68543b34991b2819fdd74c480db
I20260806 21:11:01.171336 18046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:11:01.179078 18046 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:11:01.180415 18046 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:11:01.188938 18046 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40c8e0f171e248a7a5cb8c00d73ceafb: Generated new TSK 0
I20260806 21:11:01.189903 18046 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:11:01.195156 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 17953
W20260806 21:11:01.201705 17938 connection.cc:535] client connection to 127.0.0.1:43039 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43039): Connection reset by peer (error 104)
2026-08-06T21:11:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1505 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:11:01 dist-test-slave-69rg krb5kdc[18055](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:11:01 dist-test-slave-69rg krb5kdc[18055](info): set up 2 sockets
Aug 06 21:11:01 dist-test-slave-69rg krb5kdc[18055](info): commencing operation
krb5kdc: starting...
W20260806 21:11:02.288991 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:11:02 dist-test-slave-69rg krb5kdc[18055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:11:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:11:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 21:11:02.426723 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33187
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33187
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:02.546432 18069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:02.546720 18069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:02.546768 18069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:02.550942 18069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:02.551019 18069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:02.551043 18069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:02.551059 18069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:02.551074 18069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:11:02.556497 18069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33187
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:02.557515 18069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:02.558588 18069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:02.567224 18075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:02.567294 18074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:02.567440 18069 server_base.cc:1034] running on GCE node
W20260806 21:11:02.567227 18077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:02.568123 18069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:02.569238 18069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:02.570422 18069 hybrid_clock.cc:648] HybridClock initialized: now 1786050662570394 us; error 44 us; skew 500 ppm
Aug 06 21:11:02 dist-test-slave-69rg krb5kdc[18055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050662, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:02.573297 18069 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 21:11:02.574287 18069 webserver.cc:460] Webserver started at http://127.0.0.1:37335/ using document root <none> and password file <none>
I20260806 21:11:02.574918 18069 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:02.575011 18069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:02.575237 18069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:02.577276 18069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "1b6774cbc17b49d892d535c3bf85984b"
format_stamp: "Formatted at 2026-08-06 21:11:02 on dist-test-slave-69rg"
server_key: "3553d369482a46b2da6bf6db564792cd"
server_key_iv: "04ea7c3711c7656420d9b29beaeaffc3"
server_key_version: "encryptionkey@0"
I20260806 21:11:02.577920 18069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "1b6774cbc17b49d892d535c3bf85984b"
format_stamp: "Formatted at 2026-08-06 21:11:02 on dist-test-slave-69rg"
server_key: "3553d369482a46b2da6bf6db564792cd"
server_key_iv: "04ea7c3711c7656420d9b29beaeaffc3"
server_key_version: "encryptionkey@0"
I20260806 21:11:02.581723 18069 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 21:11:02.584301 18084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:02.585681 18069 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:11:02.585875 18069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "1b6774cbc17b49d892d535c3bf85984b"
format_stamp: "Formatted at 2026-08-06 21:11:02 on dist-test-slave-69rg"
server_key: "3553d369482a46b2da6bf6db564792cd"
server_key_iv: "04ea7c3711c7656420d9b29beaeaffc3"
server_key_version: "encryptionkey@0"
I20260806 21:11:02.586001 18069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:02.590641 18069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:02.594131 18069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:02.594337 18069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:02.601846 18069 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33187
I20260806 21:11:02.601941 18136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33187 every 8 connection(s)
I20260806 21:11:02.602833 18069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:11:02.605259 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 18069
I20260806 21:11:02.605371 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:11:02.605677 28551 external_mini_cluster.cc:1428] Setting key 1f79f94362006c98f041dcf17c6db8e7
I20260806 21:11:02.605973 18137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:02.611722 18140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:02.607188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:60053 (local address 127.0.0.1:33187)
0806 21:11:02.607686 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:02.607702 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:02.607955 (+ 253us) server_negotiation.cc:407] Connection header received
0806 21:11:02.608931 (+ 976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:02.608951 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:02.609236 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:02.609523 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:02.610451 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:02.610456 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:02.610467 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:02.610479 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:02.610537 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:02.610557 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:02.610708 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:02.610817 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 21:11:02.610985 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":157,"threads_started":1}
I20260806 21:11:02.613607 18137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b6774cbc17b49d892d535c3bf85984b: Bootstrap starting.
I20260806 21:11:02.614634 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:11:02.616236 18137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b6774cbc17b49d892d535c3bf85984b: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:02.617182 18137 log.cc:824] T 00000000000000000000000000000000 P 1b6774cbc17b49d892d535c3bf85984b: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:02.618710 18140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:02.616758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:52907 (local address 127.0.0.1:33187)
0806 21:11:02.616876 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:02.616881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:02.617062 (+ 181us) server_negotiation.cc:407] Connection header received
0806 21:11:02.617142 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:02.617145 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:02.617187 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:02.617298 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:02.618276 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:02.618280 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:02.618283 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:02.618298 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:02.618328 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:02.618342 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:02.618438 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:02.618544 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 21:11:02.618587 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 21:11:02.619728 18137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b6774cbc17b49d892d535c3bf85984b: No bootstrap required, opened a new log
I20260806 21:11:02.623579 18137 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b6774cbc17b49d892d535c3bf85984b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b6774cbc17b49d892d535c3bf85984b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33187 } }
I20260806 21:11:02.623831 18137 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b6774cbc17b49d892d535c3bf85984b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:02.623870 18137 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b6774cbc17b49d892d535c3bf85984b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b6774cbc17b49d892d535c3bf85984b, State: Initialized, Role: FOLLOWER
I20260806 21:11:02.624323 18137 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b6774cbc17b49d892d535c3bf85984b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b6774cbc17b49d892d535c3bf85984b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33187 } }
I20260806 21:11:02.624516 18137 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b6774cbc17b49d892d535c3bf85984b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:11:02.624603 18137 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b6774cbc17b49d892d535c3bf85984b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:11:02.624737 18137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b6774cbc17b49d892d535c3bf85984b [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:02.626019 18137 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b6774cbc17b49d892d535c3bf85984b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b6774cbc17b49d892d535c3bf85984b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33187 } }
I20260806 21:11:02.626379 18137 leader_election.cc:302] T 00000000000000000000000000000000 P 1b6774cbc17b49d892d535c3bf85984b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b6774cbc17b49d892d535c3bf85984b; no voters:
I20260806 21:11:02.626729 18137 leader_election.cc:288] T 00000000000000000000000000000000 P 1b6774cbc17b49d892d535c3bf85984b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:11:02.626845 18148 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b6774cbc17b49d892d535c3bf85984b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:02.627092 18148 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b6774cbc17b49d892d535c3bf85984b [term 1 LEADER]: Becoming Leader. State: Replica: 1b6774cbc17b49d892d535c3bf85984b, State: Running, Role: LEADER
I20260806 21:11:02.627480 18148 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b6774cbc17b49d892d535c3bf85984b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b6774cbc17b49d892d535c3bf85984b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33187 } }
I20260806 21:11:02.628096 18137 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b6774cbc17b49d892d535c3bf85984b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:02.629259 18149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b6774cbc17b49d892d535c3bf85984b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b6774cbc17b49d892d535c3bf85984b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b6774cbc17b49d892d535c3bf85984b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33187 } } }
I20260806 21:11:02.629393 18149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b6774cbc17b49d892d535c3bf85984b [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:02.629740 18150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b6774cbc17b49d892d535c3bf85984b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b6774cbc17b49d892d535c3bf85984b. Latest consensus state: current_term: 1 leader_uuid: "1b6774cbc17b49d892d535c3bf85984b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b6774cbc17b49d892d535c3bf85984b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33187 } } }
I20260806 21:11:02.629949 18150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b6774cbc17b49d892d535c3bf85984b [sys.catalog]: This master's current role is: LEADER
W20260806 21:11:02.633002 18164 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1b6774cbc17b49d892d535c3bf85984b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:11:02.633112 18164 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:11:02.633204 18157 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:11:02.634135 18157 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:11:02.639950 18157 catalog_manager.cc:1347] Generated new cluster ID: f75b4be46f0f42eda17de662ba224f2a
I20260806 21:11:02.640046 18157 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:11:02.655176 18157 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:11:02.655529 18157 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:11:02.668303 18157 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b6774cbc17b49d892d535c3bf85984b: Generated new TSK 0
I20260806 21:11:02.669078 18157 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:11:02.672344 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 18069
2026-08-06T21:11:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1477 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:11:02 dist-test-slave-69rg krb5kdc[18169](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:11:02 dist-test-slave-69rg krb5kdc[18169](info): set up 2 sockets
Aug 06 21:11:02 dist-test-slave-69rg krb5kdc[18169](info): commencing operation
krb5kdc: starting...
W20260806 21:11:03.766044 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:11:03 dist-test-slave-69rg krb5kdc[18169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:11:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:11:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:11:03.905020 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:40987
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:34281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:40987
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:04.020493 18183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:04.020889 18183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:04.020977 18183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:04.025223 18183 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:04.025342 18183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:04.025377 18183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:04.025420 18183 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:04.025452 18183 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:11:04.031219 18183 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:34281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:40987
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:40987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:04.032436 18183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:04.033587 18183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:04.042856 18189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:04.042833 18188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:04.042984 18191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:04.043277 18183 server_base.cc:1034] running on GCE node
I20260806 21:11:04.043848 18183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:04.044919 18183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:04.046080 18183 hybrid_clock.cc:648] HybridClock initialized: now 1786050664046066 us; error 30 us; skew 500 ppm
Aug 06 21:11:04 dist-test-slave-69rg krb5kdc[18169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050664, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:04.048918 18183 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:11:04.050179 18183 webserver.cc:460] Webserver started at http://127.27.225.254:45755/ using document root <none> and password file <none>
I20260806 21:11:04.050750 18183 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:04.050818 18183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:04.050998 18183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:04.053234 18183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "806f4c82bee34fe2b92ffbf621a19e61"
format_stamp: "Formatted at 2026-08-06 21:11:04 on dist-test-slave-69rg"
server_key: "b67d5641a583113293e03ac5b930e0fe"
server_key_iv: "0453d7666f0d1863fc493ba182ae5c91"
server_key_version: "encryptionkey@0"
I20260806 21:11:04.053800 18183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "806f4c82bee34fe2b92ffbf621a19e61"
format_stamp: "Formatted at 2026-08-06 21:11:04 on dist-test-slave-69rg"
server_key: "b67d5641a583113293e03ac5b930e0fe"
server_key_iv: "0453d7666f0d1863fc493ba182ae5c91"
server_key_version: "encryptionkey@0"
I20260806 21:11:04.057971 18183 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 21:11:04.062639 18198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:04.064674 18183 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 21:11:04.064855 18183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "806f4c82bee34fe2b92ffbf621a19e61"
format_stamp: "Formatted at 2026-08-06 21:11:04 on dist-test-slave-69rg"
server_key: "b67d5641a583113293e03ac5b930e0fe"
server_key_iv: "0453d7666f0d1863fc493ba182ae5c91"
server_key_version: "encryptionkey@0"
I20260806 21:11:04.065022 18183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:04.081414 18183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:04.085161 18183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:04.085372 18183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:04.092880 18183 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:40987
I20260806 21:11:04.092978 18252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:40987 every 8 connection(s)
I20260806 21:11:04.093997 18183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:11:04.096885 18253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:04.102612 18253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 806f4c82bee34fe2b92ffbf621a19e61: Bootstrap starting.
I20260806 21:11:04.103039 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 18183
I20260806 21:11:04.103194 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:11:04.103498 28551 external_mini_cluster.cc:1428] Setting key 9c577c6b8fa93b18b9ca10ef931acad4
I20260806 21:11:04.105468 18253 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 806f4c82bee34fe2b92ffbf621a19e61: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:04.106410 18253 log.cc:824] T 00000000000000000000000000000000 P 806f4c82bee34fe2b92ffbf621a19e61: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:04.108883 18253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 806f4c82bee34fe2b92ffbf621a19e61: No bootstrap required, opened a new log
I20260806 21:11:04.113061 18253 raft_consensus.cc:348] T 00000000000000000000000000000000 P 806f4c82bee34fe2b92ffbf621a19e61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "806f4c82bee34fe2b92ffbf621a19e61" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40987 } }
I20260806 21:11:04.113358 18253 raft_consensus.cc:374] T 00000000000000000000000000000000 P 806f4c82bee34fe2b92ffbf621a19e61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:04.113411 18253 raft_consensus.cc:729] T 00000000000000000000000000000000 P 806f4c82bee34fe2b92ffbf621a19e61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 806f4c82bee34fe2b92ffbf621a19e61, State: Initialized, Role: FOLLOWER
Aug 06 21:11:04 dist-test-slave-69rg krb5kdc[18169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:11:04.113947 18253 consensus_queue.cc:260] T 00000000000000000000000000000000 P 806f4c82bee34fe2b92ffbf621a19e61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "806f4c82bee34fe2b92ffbf621a19e61" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40987 } }
I20260806 21:11:04.114125 18253 raft_consensus.cc:388] T 00000000000000000000000000000000 P 806f4c82bee34fe2b92ffbf621a19e61 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:11:04.114199 18253 raft_consensus.cc:482] T 00000000000000000000000000000000 P 806f4c82bee34fe2b92ffbf621a19e61 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:11:04.114319 18253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 806f4c82bee34fe2b92ffbf621a19e61 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:04.115257 18253 raft_consensus.cc:504] T 00000000000000000000000000000000 P 806f4c82bee34fe2b92ffbf621a19e61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "806f4c82bee34fe2b92ffbf621a19e61" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40987 } }
I20260806 21:11:04.115597 18253 leader_election.cc:302] T 00000000000000000000000000000000 P 806f4c82bee34fe2b92ffbf621a19e61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 806f4c82bee34fe2b92ffbf621a19e61; no voters:
I20260806 21:11:04.115948 18253 leader_election.cc:288] T 00000000000000000000000000000000 P 806f4c82bee34fe2b92ffbf621a19e61 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:11:04.116025 18258 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 806f4c82bee34fe2b92ffbf621a19e61 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:04.116216 18258 raft_consensus.cc:686] T 00000000000000000000000000000000 P 806f4c82bee34fe2b92ffbf621a19e61 [term 1 LEADER]: Becoming Leader. State: Replica: 806f4c82bee34fe2b92ffbf621a19e61, State: Running, Role: LEADER
I20260806 21:11:04.116503 18258 consensus_queue.cc:237] T 00000000000000000000000000000000 P 806f4c82bee34fe2b92ffbf621a19e61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "806f4c82bee34fe2b92ffbf621a19e61" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40987 } }
I20260806 21:11:04.117250 18253 sys_catalog.cc:564] T 00000000000000000000000000000000 P 806f4c82bee34fe2b92ffbf621a19e61 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:04.117640 18258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 806f4c82bee34fe2b92ffbf621a19e61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 806f4c82bee34fe2b92ffbf621a19e61. Latest consensus state: current_term: 1 leader_uuid: "806f4c82bee34fe2b92ffbf621a19e61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "806f4c82bee34fe2b92ffbf621a19e61" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40987 } } }
I20260806 21:11:04.117746 18258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 806f4c82bee34fe2b92ffbf621a19e61 [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:04.119459 18267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:11:04.119685 18259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 806f4c82bee34fe2b92ffbf621a19e61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "806f4c82bee34fe2b92ffbf621a19e61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "806f4c82bee34fe2b92ffbf621a19e61" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 40987 } } }
I20260806 21:11:04.119997 18259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 806f4c82bee34fe2b92ffbf621a19e61 [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:04.120555 18256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:04.104945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34573 (local address 127.27.225.254:40987)
0806 21:11:04.105404 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:04.105427 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:04.105474 (+ 47us) server_negotiation.cc:407] Connection header received
0806 21:11:04.106452 (+ 978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:04.106477 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:04.106930 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:04.107240 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:04.108122 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:04.109569 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:04.110526 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:04.111227 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:04.111395 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:04.114441 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:04.114454 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:04.114471 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:04.114507 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:04.116512 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:04.117117 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:04.117122 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:04.117128 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:04.117252 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:04.117699 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:04.117703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:04.117704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:04.118374 (+ 670us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:04.118592 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:04.119123 (+ 531us) server_negotiation.cc:299] Negotiation successful
0806 21:11:04.119371 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":288,"thread_start_us":119,"threads_started":1}
I20260806 21:11:04.122565 18267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:11:04.123600 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:11:04.128412 18267 catalog_manager.cc:1347] Generated new cluster ID: 152e6580ffcb4fd7af28cbbd7d0c94b1
I20260806 21:11:04.128482 18267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:11:04.135733 18256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:04.125630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57403 (local address 127.27.225.254:40987)
0806 21:11:04.125741 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:04.125745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:04.126016 (+ 271us) server_negotiation.cc:407] Connection header received
0806 21:11:04.126132 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:04.126135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:04.126179 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:04.126262 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:04.127106 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:04.127840 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:04.130363 (+ 2523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:04.130612 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:04.130719 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:04.132167 (+ 1448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:04.132170 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:04.132173 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:04.132197 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:04.133685 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:04.134245 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:04.134249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:04.134251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:04.134335 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:04.134699 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:04.134703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:04.134704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:04.134910 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:04.135104 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:04.135415 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 21:11:04.135529 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 21:11:04.144824 18267 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:11:04.146487 18267 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:11:04.154172 18267 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 806f4c82bee34fe2b92ffbf621a19e61: Generated new TSK 0
I20260806 21:11:04.155009 18267 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:11:04.165154 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 18183
W20260806 21:11:04.171617 18167 connection.cc:535] client connection to 127.27.225.254:40987 recv error: Network error: failed to read from TLS socket (remote: 127.27.225.254:40987): Connection reset by peer (error 104)
2026-08-06T21:11:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1492 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:11:04 dist-test-slave-69rg krb5kdc[18284](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:11:04 dist-test-slave-69rg krb5kdc[18284](info): set up 2 sockets
Aug 06 21:11:04 dist-test-slave-69rg krb5kdc[18284](info): commencing operation
krb5kdc: starting...
W20260806 21:11:05.230044 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:11:05 dist-test-slave-69rg krb5kdc[18284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:11:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:11:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:11:05.369426 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:45713
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:40247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:45713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:05.484309 18297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:05.484616 18297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:05.484661 18297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:05.489151 18297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:05.489240 18297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:05.489261 18297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:05.489279 18297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:05.489295 18297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:11:05.494931 18297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:40247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:45713
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:45713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:05.496062 18297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:05.497076 18297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:05.506220 18303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:05.506187 18302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:05.506183 18305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:05.506500 18297 server_base.cc:1034] running on GCE node
I20260806 21:11:05.507140 18297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:05.508340 18297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:05.509554 18297 hybrid_clock.cc:648] HybridClock initialized: now 1786050665509524 us; error 44 us; skew 500 ppm
Aug 06 21:11:05 dist-test-slave-69rg krb5kdc[18284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050665, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:05.516970 18297 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:11:05.518042 18297 webserver.cc:460] Webserver started at http://127.27.225.254:41113/ using document root <none> and password file <none>
I20260806 21:11:05.518661 18297 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:05.518769 18297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:05.519014 18297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:05.521247 18297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "c6f2cc4eb6d542a685e9dc1846274bdb"
format_stamp: "Formatted at 2026-08-06 21:11:05 on dist-test-slave-69rg"
server_key: "cbf75d39d9e853daa46ada82685a45fc"
server_key_iv: "f782ea0bde9e044cd46b8afcfc69dbde"
server_key_version: "encryptionkey@0"
I20260806 21:11:05.521942 18297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "c6f2cc4eb6d542a685e9dc1846274bdb"
format_stamp: "Formatted at 2026-08-06 21:11:05 on dist-test-slave-69rg"
server_key: "cbf75d39d9e853daa46ada82685a45fc"
server_key_iv: "f782ea0bde9e044cd46b8afcfc69dbde"
server_key_version: "encryptionkey@0"
I20260806 21:11:05.525974 18297 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260806 21:11:05.528720 18313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:05.530071 18297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:11:05.530212 18297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "c6f2cc4eb6d542a685e9dc1846274bdb"
format_stamp: "Formatted at 2026-08-06 21:11:05 on dist-test-slave-69rg"
server_key: "cbf75d39d9e853daa46ada82685a45fc"
server_key_iv: "f782ea0bde9e044cd46b8afcfc69dbde"
server_key_version: "encryptionkey@0"
I20260806 21:11:05.530336 18297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:05.543856 18297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:05.547405 18297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:05.547623 18297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:05.555157 18297 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:45713
I20260806 21:11:05.555182 18367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:45713 every 8 connection(s)
I20260806 21:11:05.556174 18297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:11:05.557304 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 18297
I20260806 21:11:05.557479 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:11:05.557801 28551 external_mini_cluster.cc:1428] Setting key e1dd7713f3c279f08e40f0a842706fd6
I20260806 21:11:05.559329 18368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:05.564960 18368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6f2cc4eb6d542a685e9dc1846274bdb: Bootstrap starting.
I20260806 21:11:05.567020 18371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:05.559494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40917 (local address 127.27.225.254:45713)
0806 21:11:05.559935 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:05.559950 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:05.559985 (+ 35us) server_negotiation.cc:407] Connection header received
0806 21:11:05.560613 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:05.560629 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:05.560963 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:05.561296 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:05.561698 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:05.562796 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:05.563706 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:05.564175 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:05.564351 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:05.565344 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:05.565349 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:05.565360 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:05.565740 (+ 380us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:05.565794 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:05.565872 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:05.566017 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:05.566167 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 21:11:05.566309 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":140,"threads_started":1}
I20260806 21:11:05.567384 18368 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6f2cc4eb6d542a685e9dc1846274bdb: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:05.568200 18368 log.cc:824] T 00000000000000000000000000000000 P c6f2cc4eb6d542a685e9dc1846274bdb: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:05.570075 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:11:05.570721 18368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6f2cc4eb6d542a685e9dc1846274bdb: No bootstrap required, opened a new log
I20260806 21:11:05.575335 18368 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6f2cc4eb6d542a685e9dc1846274bdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6f2cc4eb6d542a685e9dc1846274bdb" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45713 } }
I20260806 21:11:05.575672 18368 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6f2cc4eb6d542a685e9dc1846274bdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:05.575764 18368 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6f2cc4eb6d542a685e9dc1846274bdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6f2cc4eb6d542a685e9dc1846274bdb, State: Initialized, Role: FOLLOWER
I20260806 21:11:05.576265 18368 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6f2cc4eb6d542a685e9dc1846274bdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6f2cc4eb6d542a685e9dc1846274bdb" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45713 } }
I20260806 21:11:05.576467 18368 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6f2cc4eb6d542a685e9dc1846274bdb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:11:05.576579 18368 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6f2cc4eb6d542a685e9dc1846274bdb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:11:05.576736 18368 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6f2cc4eb6d542a685e9dc1846274bdb [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:05.576778 18371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:05.572226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54195 (local address 127.27.225.254:45713)
0806 21:11:05.572343 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:05.572347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:05.572580 (+ 233us) server_negotiation.cc:407] Connection header received
0806 21:11:05.572702 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:05.572704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:05.572744 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:05.572810 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:05.573166 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:05.573686 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:05.574864 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:05.575089 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:05.575168 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:05.576169 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:05.576172 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:05.576174 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:05.576184 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:05.576209 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:05.576219 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:05.576318 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:05.576552 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 21:11:05.576604 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 21:11:05.578055 18368 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6f2cc4eb6d542a685e9dc1846274bdb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6f2cc4eb6d542a685e9dc1846274bdb" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45713 } }
I20260806 21:11:05.578436 18368 leader_election.cc:302] T 00000000000000000000000000000000 P c6f2cc4eb6d542a685e9dc1846274bdb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6f2cc4eb6d542a685e9dc1846274bdb; no voters:
I20260806 21:11:05.578758 18368 leader_election.cc:288] T 00000000000000000000000000000000 P c6f2cc4eb6d542a685e9dc1846274bdb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:11:05.578884 18378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6f2cc4eb6d542a685e9dc1846274bdb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:05.579135 18378 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6f2cc4eb6d542a685e9dc1846274bdb [term 1 LEADER]: Becoming Leader. State: Replica: c6f2cc4eb6d542a685e9dc1846274bdb, State: Running, Role: LEADER
I20260806 21:11:05.579486 18378 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6f2cc4eb6d542a685e9dc1846274bdb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6f2cc4eb6d542a685e9dc1846274bdb" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45713 } }
I20260806 21:11:05.580081 18368 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6f2cc4eb6d542a685e9dc1846274bdb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:05.582958 18379 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6f2cc4eb6d542a685e9dc1846274bdb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6f2cc4eb6d542a685e9dc1846274bdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6f2cc4eb6d542a685e9dc1846274bdb" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45713 } } }
I20260806 21:11:05.583024 18381 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6f2cc4eb6d542a685e9dc1846274bdb [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6f2cc4eb6d542a685e9dc1846274bdb. Latest consensus state: current_term: 1 leader_uuid: "c6f2cc4eb6d542a685e9dc1846274bdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6f2cc4eb6d542a685e9dc1846274bdb" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45713 } } }
I20260806 21:11:05.583367 18379 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6f2cc4eb6d542a685e9dc1846274bdb [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:05.583367 18381 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6f2cc4eb6d542a685e9dc1846274bdb [sys.catalog]: This master's current role is: LEADER
W20260806 21:11:05.585873 18394 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c6f2cc4eb6d542a685e9dc1846274bdb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:11:05.585975 18394 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:11:05.586053 18395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:11:05.586794 18395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:11:05.593078 18395 catalog_manager.cc:1347] Generated new cluster ID: bf5eae0002e3410c9fd351f0939d52ff
I20260806 21:11:05.593176 18395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:11:05.609326 18395 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:11:05.610723 18395 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:11:05.616698 18395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6f2cc4eb6d542a685e9dc1846274bdb: Generated new TSK 0
I20260806 21:11:05.617560 18395 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:11:05.628401 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 18297
2026-08-06T21:11:05Z chronyd exiting
W20260806 21:11:05.637531 18282 connection.cc:535] client connection to 127.27.225.254:45713 recv error: Network error: failed to read from TLS socket (remote: 127.27.225.254:45713): Connection reset by peer (error 104)
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1464 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:11:05 dist-test-slave-69rg krb5kdc[18400](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:11:05 dist-test-slave-69rg krb5kdc[18400](info): set up 2 sockets
Aug 06 21:11:05 dist-test-slave-69rg krb5kdc[18400](info): commencing operation
krb5kdc: starting...
W20260806 21:11:06.713747 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:11:06 dist-test-slave-69rg krb5kdc[18400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:11:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:11:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:11:06.858410 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:41583
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:35919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:41583
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:06.975419 18414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:06.975795 18414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:06.975888 18414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:06.980134 18414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:06.980273 18414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:06.980316 18414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:06.980372 18414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:06.980410 18414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:11:06.986184 18414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:35919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:41583
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:41583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:06.987470 18414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:06.988557 18414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:06.997526 18420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:06.997529 18422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:06.997555 18419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:06.997776 18414 server_base.cc:1034] running on GCE node
I20260806 21:11:06.998505 18414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:06.999679 18414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:07.000888 18414 hybrid_clock.cc:648] HybridClock initialized: now 1786050667000847 us; error 62 us; skew 500 ppm
Aug 06 21:11:07 dist-test-slave-69rg krb5kdc[18400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050667, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:07.004112 18414 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:11:07.005223 18414 webserver.cc:460] Webserver started at http://127.27.225.254:38647/ using document root <none> and password file <none>
I20260806 21:11:07.005908 18414 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:07.006016 18414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:07.006275 18414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:07.008668 18414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "35d95185c24f4f52a00ca27cd2d5a85b"
format_stamp: "Formatted at 2026-08-06 21:11:07 on dist-test-slave-69rg"
server_key: "83de11ea5988a15173d6afd74d562b08"
server_key_iv: "a5d4c9881e9f258a048e07178b4275a7"
server_key_version: "encryptionkey@0"
I20260806 21:11:07.009373 18414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "35d95185c24f4f52a00ca27cd2d5a85b"
format_stamp: "Formatted at 2026-08-06 21:11:07 on dist-test-slave-69rg"
server_key: "83de11ea5988a15173d6afd74d562b08"
server_key_iv: "a5d4c9881e9f258a048e07178b4275a7"
server_key_version: "encryptionkey@0"
I20260806 21:11:07.013744 18414 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260806 21:11:07.016839 18429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:07.018286 18414 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:11:07.018457 18414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "35d95185c24f4f52a00ca27cd2d5a85b"
format_stamp: "Formatted at 2026-08-06 21:11:07 on dist-test-slave-69rg"
server_key: "83de11ea5988a15173d6afd74d562b08"
server_key_iv: "a5d4c9881e9f258a048e07178b4275a7"
server_key_version: "encryptionkey@0"
I20260806 21:11:07.018618 18414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:07.023820 18414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:07.027599 18414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:07.027833 18414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:07.036145 18414 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:41583
I20260806 21:11:07.036159 18484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:41583 every 8 connection(s)
I20260806 21:11:07.037298 18414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:11:07.040331 18485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:07.046497 18485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35d95185c24f4f52a00ca27cd2d5a85b: Bootstrap starting.
I20260806 21:11:07.046928 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 18414
I20260806 21:11:07.047083 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:11:07.047592 28551 external_mini_cluster.cc:1428] Setting key a9f43bc073a28b7b59fc85fd677c0122
I20260806 21:11:07.049654 18485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35d95185c24f4f52a00ca27cd2d5a85b: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:07.050501 18485 log.cc:824] T 00000000000000000000000000000000 P 35d95185c24f4f52a00ca27cd2d5a85b: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:07.052953 18485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35d95185c24f4f52a00ca27cd2d5a85b: No bootstrap required, opened a new log
I20260806 21:11:07.053556 18489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:07.049217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40755 (local address 127.27.225.254:41583)
0806 21:11:07.049701 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:07.049718 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:07.049754 (+ 36us) server_negotiation.cc:407] Connection header received
0806 21:11:07.050543 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:07.050562 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:07.050901 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:07.051188 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:07.052091 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:07.052097 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:07.052112 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:07.052124 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:07.052191 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:07.052227 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:07.052346 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:07.052467 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 21:11:07.052617 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":173,"threads_started":1}
I20260806 21:11:07.056620 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:11:07.057300 18485 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35d95185c24f4f52a00ca27cd2d5a85b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35d95185c24f4f52a00ca27cd2d5a85b" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41583 } }
I20260806 21:11:07.057531 18485 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35d95185c24f4f52a00ca27cd2d5a85b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:07.057591 18485 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35d95185c24f4f52a00ca27cd2d5a85b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35d95185c24f4f52a00ca27cd2d5a85b, State: Initialized, Role: FOLLOWER
I20260806 21:11:07.058096 18485 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35d95185c24f4f52a00ca27cd2d5a85b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35d95185c24f4f52a00ca27cd2d5a85b" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41583 } }
I20260806 21:11:07.058254 18485 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35d95185c24f4f52a00ca27cd2d5a85b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:11:07.058372 18485 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35d95185c24f4f52a00ca27cd2d5a85b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:11:07.058491 18485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35d95185c24f4f52a00ca27cd2d5a85b [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:07.059769 18485 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35d95185c24f4f52a00ca27cd2d5a85b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35d95185c24f4f52a00ca27cd2d5a85b" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41583 } }
I20260806 21:11:07.060142 18485 leader_election.cc:302] T 00000000000000000000000000000000 P 35d95185c24f4f52a00ca27cd2d5a85b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35d95185c24f4f52a00ca27cd2d5a85b; no voters:
I20260806 21:11:07.060467 18485 leader_election.cc:288] T 00000000000000000000000000000000 P 35d95185c24f4f52a00ca27cd2d5a85b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:11:07.060588 18496 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35d95185c24f4f52a00ca27cd2d5a85b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:07.060855 18496 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35d95185c24f4f52a00ca27cd2d5a85b [term 1 LEADER]: Becoming Leader. State: Replica: 35d95185c24f4f52a00ca27cd2d5a85b, State: Running, Role: LEADER
I20260806 21:11:07.061093 18489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:07.059150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58213 (local address 127.27.225.254:41583)
0806 21:11:07.059263 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:07.059268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:07.059525 (+ 257us) server_negotiation.cc:407] Connection header received
0806 21:11:07.059612 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:07.059617 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:07.059664 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:07.059756 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:07.060658 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:07.060664 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:07.060669 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:07.060683 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:07.060715 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:07.060731 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:07.060821 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:07.060944 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 21:11:07.060982 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 21:11:07.061228 18496 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35d95185c24f4f52a00ca27cd2d5a85b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35d95185c24f4f52a00ca27cd2d5a85b" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41583 } }
I20260806 21:11:07.061902 18485 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35d95185c24f4f52a00ca27cd2d5a85b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:07.063298 18497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35d95185c24f4f52a00ca27cd2d5a85b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35d95185c24f4f52a00ca27cd2d5a85b. Latest consensus state: current_term: 1 leader_uuid: "35d95185c24f4f52a00ca27cd2d5a85b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35d95185c24f4f52a00ca27cd2d5a85b" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41583 } } }
I20260806 21:11:07.063313 18498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35d95185c24f4f52a00ca27cd2d5a85b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35d95185c24f4f52a00ca27cd2d5a85b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35d95185c24f4f52a00ca27cd2d5a85b" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 41583 } } }
I20260806 21:11:07.063444 18497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35d95185c24f4f52a00ca27cd2d5a85b [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:07.063459 18498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35d95185c24f4f52a00ca27cd2d5a85b [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:07.063849 18506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:11:07.067134 18506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:11:07.073144 18506 catalog_manager.cc:1347] Generated new cluster ID: 7b46bbaec6834e5eaac0ef4b700589c6
I20260806 21:11:07.073239 18506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:11:07.086000 18506 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:11:07.086397 18506 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:11:07.093392 18506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35d95185c24f4f52a00ca27cd2d5a85b: Generated new TSK 0
I20260806 21:11:07.094211 18506 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:11:07.106757 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 18414
2026-08-06T21:11:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1476 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:11:07 dist-test-slave-69rg krb5kdc[18517](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:11:07 dist-test-slave-69rg krb5kdc[18517](info): set up 2 sockets
Aug 06 21:11:07 dist-test-slave-69rg krb5kdc[18517](info): commencing operation
krb5kdc: starting...
W20260806 21:11:08.225692 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:11:08 dist-test-slave-69rg krb5kdc[18517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:11:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:11:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:11:08.374724 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:45563
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:37999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:45563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:08.497496 18530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:08.497866 18530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:08.497967 18530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:08.502199 18530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:08.502314 18530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:08.502354 18530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:08.502395 18530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:08.502427 18530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:11:08.508080 18530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:37999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:45563
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:45563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:08.509394 18530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:08.510507 18530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:08.519351 18536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:08.519343 18538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:08.519397 18530 server_base.cc:1034] running on GCE node
W20260806 21:11:08.519343 18535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:08.520287 18530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:08.521507 18530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:08.522701 18530 hybrid_clock.cc:648] HybridClock initialized: now 1786050668522664 us; error 52 us; skew 500 ppm
Aug 06 21:11:08 dist-test-slave-69rg krb5kdc[18517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050668, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:08.526079 18530 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:11:08.527171 18530 webserver.cc:460] Webserver started at http://127.27.225.254:45791/ using document root <none> and password file <none>
I20260806 21:11:08.527932 18530 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:08.528044 18530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:08.528312 18530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:08.530925 18530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "91c955deba3942a8a7d247e93f354f3a"
format_stamp: "Formatted at 2026-08-06 21:11:08 on dist-test-slave-69rg"
server_key: "c7ffa86406b26cfd9a3f25ccd3aebe3f"
server_key_iv: "2914821e3e5c4f97c69c8c1876ec8cc6"
server_key_version: "encryptionkey@0"
I20260806 21:11:08.531603 18530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "91c955deba3942a8a7d247e93f354f3a"
format_stamp: "Formatted at 2026-08-06 21:11:08 on dist-test-slave-69rg"
server_key: "c7ffa86406b26cfd9a3f25ccd3aebe3f"
server_key_iv: "2914821e3e5c4f97c69c8c1876ec8cc6"
server_key_version: "encryptionkey@0"
I20260806 21:11:08.536062 18530 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:11:08.538954 18545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:08.540334 18530 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:11:08.540529 18530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "91c955deba3942a8a7d247e93f354f3a"
format_stamp: "Formatted at 2026-08-06 21:11:08 on dist-test-slave-69rg"
server_key: "c7ffa86406b26cfd9a3f25ccd3aebe3f"
server_key_iv: "2914821e3e5c4f97c69c8c1876ec8cc6"
server_key_version: "encryptionkey@0"
I20260806 21:11:08.540688 18530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:08.553942 18530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:08.557195 18530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:08.557440 18530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:08.565449 18530 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:45563
I20260806 21:11:08.565481 18599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:45563 every 8 connection(s)
I20260806 21:11:08.566537 18530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:11:08.569288 18600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:08.573313 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 18530
I20260806 21:11:08.573484 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:11:08.573845 28551 external_mini_cluster.cc:1428] Setting key edd5824e2c9846d7b0150fe6f9849415
I20260806 21:11:08.575760 18600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91c955deba3942a8a7d247e93f354f3a: Bootstrap starting.
I20260806 21:11:08.578119 18600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91c955deba3942a8a7d247e93f354f3a: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:08.578972 18600 log.cc:824] T 00000000000000000000000000000000 P 91c955deba3942a8a7d247e93f354f3a: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:08.581185 18600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91c955deba3942a8a7d247e93f354f3a: No bootstrap required, opened a new log
Aug 06 21:11:08 dist-test-slave-69rg krb5kdc[18517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:11:08.585575 18600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91c955deba3942a8a7d247e93f354f3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c955deba3942a8a7d247e93f354f3a" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45563 } }
I20260806 21:11:08.585943 18600 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91c955deba3942a8a7d247e93f354f3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:08.586063 18600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91c955deba3942a8a7d247e93f354f3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c955deba3942a8a7d247e93f354f3a, State: Initialized, Role: FOLLOWER
I20260806 21:11:08.586625 18600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91c955deba3942a8a7d247e93f354f3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c955deba3942a8a7d247e93f354f3a" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45563 } }
I20260806 21:11:08.586781 18600 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91c955deba3942a8a7d247e93f354f3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:11:08.586886 18600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91c955deba3942a8a7d247e93f354f3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:11:08.586992 18600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91c955deba3942a8a7d247e93f354f3a [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:08.587968 18600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91c955deba3942a8a7d247e93f354f3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c955deba3942a8a7d247e93f354f3a" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45563 } }
I20260806 21:11:08.588351 18600 leader_election.cc:302] T 00000000000000000000000000000000 P 91c955deba3942a8a7d247e93f354f3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91c955deba3942a8a7d247e93f354f3a; no voters:
I20260806 21:11:08.588719 18600 leader_election.cc:288] T 00000000000000000000000000000000 P 91c955deba3942a8a7d247e93f354f3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:11:08.588809 18606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91c955deba3942a8a7d247e93f354f3a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:08.589021 18606 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91c955deba3942a8a7d247e93f354f3a [term 1 LEADER]: Becoming Leader. State: Replica: 91c955deba3942a8a7d247e93f354f3a, State: Running, Role: LEADER
I20260806 21:11:08.589407 18606 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91c955deba3942a8a7d247e93f354f3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c955deba3942a8a7d247e93f354f3a" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45563 } }
I20260806 21:11:08.590222 18600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91c955deba3942a8a7d247e93f354f3a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:08.591279 18607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91c955deba3942a8a7d247e93f354f3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91c955deba3942a8a7d247e93f354f3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c955deba3942a8a7d247e93f354f3a" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45563 } } }
I20260806 21:11:08.591444 18607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91c955deba3942a8a7d247e93f354f3a [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:08.591843 18608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91c955deba3942a8a7d247e93f354f3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91c955deba3942a8a7d247e93f354f3a. Latest consensus state: current_term: 1 leader_uuid: "91c955deba3942a8a7d247e93f354f3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91c955deba3942a8a7d247e93f354f3a" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 45563 } } }
I20260806 21:11:08.592006 18608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91c955deba3942a8a7d247e93f354f3a [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:08.592078 18603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:08.575484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52967 (local address 127.27.225.254:45563)
0806 21:11:08.576028 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:08.576046 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:08.576085 (+ 39us) server_negotiation.cc:407] Connection header received
0806 21:11:08.576845 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:08.576863 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:08.577233 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:08.577602 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:08.578656 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:08.580379 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:08.581359 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:08.581982 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:08.582171 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:08.585331 (+ 3160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:08.585337 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:08.585354 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:08.585391 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:08.587635 (+ 2244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:08.588182 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:08.588189 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:08.588195 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:08.588335 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:08.588684 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:08.588688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:08.588690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:08.589360 (+ 670us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:08.590339 (+ 979us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:08.590515 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 21:11:08.590894 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":153,"threads_started":1}
I20260806 21:11:08.592777 18615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:11:08.595816 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:11:08.596217 18615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:11:08.604385 18615 catalog_manager.cc:1347] Generated new cluster ID: ae6cef6ffe3f4724a81e83f31e8dd8ca
I20260806 21:11:08.604488 18615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:11:08.610961 18603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:08.598212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36145 (local address 127.27.225.254:45563)
0806 21:11:08.598470 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:08.598477 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:08.600286 (+ 1809us) server_negotiation.cc:407] Connection header received
0806 21:11:08.600415 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:08.600418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:08.600479 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:08.600586 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:08.601510 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:08.602429 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:08.603269 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:08.603542 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:08.603662 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:08.606991 (+ 3329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:08.606997 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:08.607001 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:08.607037 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:08.608960 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:08.609478 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:08.609482 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:08.609484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:08.609593 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:08.609948 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:08.609952 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:08.609954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:08.610207 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:08.610355 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:08.610624 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 21:11:08.610776 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181}
I20260806 21:11:08.612504 18615 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:11:08.613479 18615 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:11:08.620388 18615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91c955deba3942a8a7d247e93f354f3a: Generated new TSK 0
I20260806 21:11:08.621119 18615 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:11:08.628962 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 18530
2026-08-06T21:11:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1521 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:11:08 dist-test-slave-69rg krb5kdc[18633](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:11:08 dist-test-slave-69rg krb5kdc[18633](info): set up 2 sockets
Aug 06 21:11:08 dist-test-slave-69rg krb5kdc[18633](info): commencing operation
krb5kdc: starting...
W20260806 21:11:09.745805 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:11:09 dist-test-slave-69rg krb5kdc[18633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:11:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:11:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:11:09.883383 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:36551
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:39435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:36551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:09.996960 18646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:09.997313 18646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:09.997392 18646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:10.001716 18646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:10.001818 18646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:10.001886 18646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:10.001928 18646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:10.001961 18646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:11:10.007663 18646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:39435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:36551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:36551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:10.009016 18646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:10.010241 18646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:10.019827 18652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:10.019865 18651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:10.020120 18646 server_base.cc:1034] running on GCE node
W20260806 21:11:10.019865 18654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:10.020874 18646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:10.022296 18646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:10.023540 18646 hybrid_clock.cc:648] HybridClock initialized: now 1786050670023500 us; error 54 us; skew 500 ppm
Aug 06 21:11:10 dist-test-slave-69rg krb5kdc[18633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050670, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:10.027295 18646 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:11:10.028477 18646 webserver.cc:460] Webserver started at http://127.27.225.254:37489/ using document root <none> and password file <none>
I20260806 21:11:10.029143 18646 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:10.029251 18646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:10.029685 18646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:10.032178 18646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "d7b7713361434fe5ad523ac03aa1220c"
format_stamp: "Formatted at 2026-08-06 21:11:10 on dist-test-slave-69rg"
server_key: "b656a793c1faff0162b7303199620200"
server_key_iv: "1809706b66a3251d853c5e8334c72556"
server_key_version: "encryptionkey@0"
I20260806 21:11:10.032833 18646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "d7b7713361434fe5ad523ac03aa1220c"
format_stamp: "Formatted at 2026-08-06 21:11:10 on dist-test-slave-69rg"
server_key: "b656a793c1faff0162b7303199620200"
server_key_iv: "1809706b66a3251d853c5e8334c72556"
server_key_version: "encryptionkey@0"
I20260806 21:11:10.037046 18646 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260806 21:11:10.039806 18661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:10.041196 18646 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 21:11:10.041365 18646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "d7b7713361434fe5ad523ac03aa1220c"
format_stamp: "Formatted at 2026-08-06 21:11:10 on dist-test-slave-69rg"
server_key: "b656a793c1faff0162b7303199620200"
server_key_iv: "1809706b66a3251d853c5e8334c72556"
server_key_version: "encryptionkey@0"
I20260806 21:11:10.041501 18646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:10.056181 18646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:10.060719 18646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:10.061038 18646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:10.071866 18646 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:36551
I20260806 21:11:10.071885 18715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:36551 every 8 connection(s)
I20260806 21:11:10.073014 18646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:11:10.076205 18716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:10.081625 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 18646
I20260806 21:11:10.081843 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:11:10.082240 28551 external_mini_cluster.cc:1428] Setting key 9c7c8db9ebd0d52b489d1a1bb348282a
I20260806 21:11:10.082872 18716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7b7713361434fe5ad523ac03aa1220c: Bootstrap starting.
I20260806 21:11:10.085988 18716 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7b7713361434fe5ad523ac03aa1220c: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:10.086978 18716 log.cc:824] T 00000000000000000000000000000000 P d7b7713361434fe5ad523ac03aa1220c: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:10.089344 18716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7b7713361434fe5ad523ac03aa1220c: No bootstrap required, opened a new log
I20260806 21:11:10.093490 18719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:10.083885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39115 (local address 127.27.225.254:36551)
0806 21:11:10.084400 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:10.084430 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:10.084480 (+ 50us) server_negotiation.cc:407] Connection header received
0806 21:11:10.085496 (+ 1016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:10.085524 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:10.086087 (+ 563us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:10.086530 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:10.086999 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:10.088376 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:10.089337 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:10.090096 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:10.090260 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:10.091185 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:10.091192 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:10.091210 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:10.091764 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:10.091843 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:10.091873 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:10.091971 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:10.092119 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 21:11:10.092352 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":148,"threads_started":1}
I20260806 21:11:10.094432 18716 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7b7713361434fe5ad523ac03aa1220c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b7713361434fe5ad523ac03aa1220c" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 36551 } }
I20260806 21:11:10.094730 18716 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7b7713361434fe5ad523ac03aa1220c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:10.094786 18716 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7b7713361434fe5ad523ac03aa1220c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7b7713361434fe5ad523ac03aa1220c, State: Initialized, Role: FOLLOWER
I20260806 21:11:10.095286 18716 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7b7713361434fe5ad523ac03aa1220c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b7713361434fe5ad523ac03aa1220c" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 36551 } }
I20260806 21:11:10.095449 18716 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7b7713361434fe5ad523ac03aa1220c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:11:10.095522 18716 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7b7713361434fe5ad523ac03aa1220c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:11:10.095628 18716 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7b7713361434fe5ad523ac03aa1220c [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:10.096740 18716 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7b7713361434fe5ad523ac03aa1220c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b7713361434fe5ad523ac03aa1220c" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 36551 } }
I20260806 21:11:10.096921 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:11:10.097124 18716 leader_election.cc:302] T 00000000000000000000000000000000 P d7b7713361434fe5ad523ac03aa1220c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7b7713361434fe5ad523ac03aa1220c; no voters:
I20260806 21:11:10.097507 18716 leader_election.cc:288] T 00000000000000000000000000000000 P d7b7713361434fe5ad523ac03aa1220c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:11:10.097591 18721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7b7713361434fe5ad523ac03aa1220c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:10.097851 18721 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7b7713361434fe5ad523ac03aa1220c [term 1 LEADER]: Becoming Leader. State: Replica: d7b7713361434fe5ad523ac03aa1220c, State: Running, Role: LEADER
I20260806 21:11:10.098237 18721 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7b7713361434fe5ad523ac03aa1220c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b7713361434fe5ad523ac03aa1220c" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 36551 } }
I20260806 21:11:10.098949 18716 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7b7713361434fe5ad523ac03aa1220c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:10.099864 18722 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7b7713361434fe5ad523ac03aa1220c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7b7713361434fe5ad523ac03aa1220c. Latest consensus state: current_term: 1 leader_uuid: "d7b7713361434fe5ad523ac03aa1220c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b7713361434fe5ad523ac03aa1220c" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 36551 } } }
I20260806 21:11:10.100018 18722 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7b7713361434fe5ad523ac03aa1220c [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:10.100284 18722 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7b7713361434fe5ad523ac03aa1220c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7b7713361434fe5ad523ac03aa1220c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b7713361434fe5ad523ac03aa1220c" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 36551 } } }
I20260806 21:11:10.100402 18722 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7b7713361434fe5ad523ac03aa1220c [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:10.100871 18735 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:11:10.104401 18735 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:11:10.105759 18719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:10.100012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49717 (local address 127.27.225.254:36551)
0806 21:11:10.100164 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:10.100169 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:10.100461 (+ 292us) server_negotiation.cc:407] Connection header received
0806 21:11:10.100674 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:10.100679 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:10.100734 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:10.100788 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:10.101310 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:10.102295 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:10.103273 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:10.103687 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:10.103796 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:10.105158 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:10.105162 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:10.105167 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:10.105187 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:10.105223 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:10.105242 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:10.105329 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:10.105511 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 21:11:10.105577 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 21:11:10.112161 18735 catalog_manager.cc:1347] Generated new cluster ID: 8d1b55838fe44b0ba9ade0e0a1aa3642
I20260806 21:11:10.112322 18735 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:11:10.123261 18735 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:11:10.124326 18735 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:11:10.129904 18735 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7b7713361434fe5ad523ac03aa1220c: Generated new TSK 0
I20260806 21:11:10.130587 18735 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:11:10.143723 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 18646
2026-08-06T21:11:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1515 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:11:10 dist-test-slave-69rg krb5kdc[18747](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:11:10 dist-test-slave-69rg krb5kdc[18747](info): set up 2 sockets
Aug 06 21:11:10 dist-test-slave-69rg krb5kdc[18747](info): commencing operation
krb5kdc: starting...
W20260806 21:11:11.209641 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:11:11 dist-test-slave-69rg krb5kdc[18747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:11:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:11:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:11:11.367010 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:36009
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:44335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:36009
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:11.488430 18760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:11.488811 18760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:11.488899 18760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:11.493283 18760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:11.493423 18760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:11.493460 18760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:11.493500 18760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:11.493556 18760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:11:11.499553 18760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:44335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:36009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:36009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:11.500829 18760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:11.501955 18760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:11.512053 18765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:11.512094 18768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:11.512094 18766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:11.512436 18760 server_base.cc:1034] running on GCE node
I20260806 21:11:11.513207 18760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:11.514808 18760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:11.516069 18760 hybrid_clock.cc:648] HybridClock initialized: now 1786050671516045 us; error 44 us; skew 500 ppm
Aug 06 21:11:11 dist-test-slave-69rg krb5kdc[18747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050671, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:11.524125 18760 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:11:11.525209 18760 webserver.cc:460] Webserver started at http://127.27.225.254:44189/ using document root <none> and password file <none>
I20260806 21:11:11.525876 18760 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:11.525985 18760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:11.526232 18760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:11.528399 18760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "4b9fe55e3eaf441abcee9e7ac0c67fb9"
format_stamp: "Formatted at 2026-08-06 21:11:11 on dist-test-slave-69rg"
server_key: "343e5fecc5bb2126c5edbfc155088825"
server_key_iv: "64d92a205cef6f69357ac41e036a53c6"
server_key_version: "encryptionkey@0"
I20260806 21:11:11.529006 18760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "4b9fe55e3eaf441abcee9e7ac0c67fb9"
format_stamp: "Formatted at 2026-08-06 21:11:11 on dist-test-slave-69rg"
server_key: "343e5fecc5bb2126c5edbfc155088825"
server_key_iv: "64d92a205cef6f69357ac41e036a53c6"
server_key_version: "encryptionkey@0"
I20260806 21:11:11.533121 18760 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:11:11.536039 18775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:11.537436 18760 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:11:11.537598 18760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "4b9fe55e3eaf441abcee9e7ac0c67fb9"
format_stamp: "Formatted at 2026-08-06 21:11:11 on dist-test-slave-69rg"
server_key: "343e5fecc5bb2126c5edbfc155088825"
server_key_iv: "64d92a205cef6f69357ac41e036a53c6"
server_key_version: "encryptionkey@0"
I20260806 21:11:11.537732 18760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:11.543146 18760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:11.546859 18760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:11.547098 18760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:11.555251 18760 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:36009
I20260806 21:11:11.555406 18829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:36009 every 8 connection(s)
I20260806 21:11:11.556660 18760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:11:11.559978 18830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:11.566030 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 18760
I20260806 21:11:11.566159 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:11:11.566249 18830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b9fe55e3eaf441abcee9e7ac0c67fb9: Bootstrap starting.
I20260806 21:11:11.566519 28551 external_mini_cluster.cc:1428] Setting key 1e1475c6ef910b0cefc795eb7f22a20f
I20260806 21:11:11.569350 18830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b9fe55e3eaf441abcee9e7ac0c67fb9: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:11.570374 18830 log.cc:824] T 00000000000000000000000000000000 P 4b9fe55e3eaf441abcee9e7ac0c67fb9: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:11.573035 18830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b9fe55e3eaf441abcee9e7ac0c67fb9: No bootstrap required, opened a new log
I20260806 21:11:11.573096 18833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:11.567877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46495 (local address 127.27.225.254:36009)
0806 21:11:11.568386 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:11.568405 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:11.568451 (+ 46us) server_negotiation.cc:407] Connection header received
0806 21:11:11.569428 (+ 977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:11.569458 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:11.569891 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:11.570309 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:11.571628 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:11.571635 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:11.571656 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:11.571673 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:11.571742 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:11.571765 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:11.571914 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:11.572032 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 21:11:11.572267 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":138,"threads_started":1}
I20260806 21:11:11.576778 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:11:11.578203 18830 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b9fe55e3eaf441abcee9e7ac0c67fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9fe55e3eaf441abcee9e7ac0c67fb9" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 36009 } }
I20260806 21:11:11.578562 18830 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b9fe55e3eaf441abcee9e7ac0c67fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:11.578665 18830 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b9fe55e3eaf441abcee9e7ac0c67fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b9fe55e3eaf441abcee9e7ac0c67fb9, State: Initialized, Role: FOLLOWER
I20260806 21:11:11.579229 18830 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b9fe55e3eaf441abcee9e7ac0c67fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9fe55e3eaf441abcee9e7ac0c67fb9" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 36009 } }
I20260806 21:11:11.579419 18830 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b9fe55e3eaf441abcee9e7ac0c67fb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:11:11.579505 18830 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b9fe55e3eaf441abcee9e7ac0c67fb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:11:11.579847 18830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b9fe55e3eaf441abcee9e7ac0c67fb9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:11.581722 18830 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b9fe55e3eaf441abcee9e7ac0c67fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9fe55e3eaf441abcee9e7ac0c67fb9" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 36009 } }
I20260806 21:11:11.581962 18833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:11.579736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60729 (local address 127.27.225.254:36009)
0806 21:11:11.579900 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:11.579904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:11.580199 (+ 295us) server_negotiation.cc:407] Connection header received
0806 21:11:11.580502 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:11.580505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:11.580545 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:11.580627 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:11.581439 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:11.581442 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:11.581443 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:11.581455 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:11.581484 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:11.581494 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:11.581571 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:11.581690 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 21:11:11.581725 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 21:11:11.582291 18830 leader_election.cc:302] T 00000000000000000000000000000000 P 4b9fe55e3eaf441abcee9e7ac0c67fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b9fe55e3eaf441abcee9e7ac0c67fb9; no voters:
I20260806 21:11:11.582744 18830 leader_election.cc:288] T 00000000000000000000000000000000 P 4b9fe55e3eaf441abcee9e7ac0c67fb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:11:11.583014 18840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b9fe55e3eaf441abcee9e7ac0c67fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:11.583307 18840 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b9fe55e3eaf441abcee9e7ac0c67fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 4b9fe55e3eaf441abcee9e7ac0c67fb9, State: Running, Role: LEADER
I20260806 21:11:11.583652 18840 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b9fe55e3eaf441abcee9e7ac0c67fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9fe55e3eaf441abcee9e7ac0c67fb9" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 36009 } }
I20260806 21:11:11.584121 18830 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b9fe55e3eaf441abcee9e7ac0c67fb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:11.585373 18841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b9fe55e3eaf441abcee9e7ac0c67fb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b9fe55e3eaf441abcee9e7ac0c67fb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9fe55e3eaf441abcee9e7ac0c67fb9" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 36009 } } }
I20260806 21:11:11.585525 18841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b9fe55e3eaf441abcee9e7ac0c67fb9 [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:11.586015 18849 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:11:11.586032 18842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b9fe55e3eaf441abcee9e7ac0c67fb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b9fe55e3eaf441abcee9e7ac0c67fb9. Latest consensus state: current_term: 1 leader_uuid: "4b9fe55e3eaf441abcee9e7ac0c67fb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9fe55e3eaf441abcee9e7ac0c67fb9" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 36009 } } }
I20260806 21:11:11.586156 18842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b9fe55e3eaf441abcee9e7ac0c67fb9 [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:11.589390 18849 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:11:11.595487 18849 catalog_manager.cc:1347] Generated new cluster ID: 6c25974cd7a24fe6923adab598e40378
I20260806 21:11:11.595599 18849 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:11:11.623003 18849 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:11:11.623435 18849 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:11:11.638089 18849 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b9fe55e3eaf441abcee9e7ac0c67fb9: Generated new TSK 0
I20260806 21:11:11.638995 18849 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:11:11.644587 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 18760
2026-08-06T21:11:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1501 ms)
[----------] 18 tests from AuthTokenIssuingTest (26880 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:11:11 dist-test-slave-69rg krb5kdc[18862](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:11:11 dist-test-slave-69rg krb5kdc[18862](info): set up 2 sockets
Aug 06 21:11:11 dist-test-slave-69rg krb5kdc[18862](info): commencing operation
krb5kdc: starting...
W20260806 21:11:12.742131 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:11:12 dist-test-slave-69rg krb5kdc[18862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:11:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:11:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:11:12.886417 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:42013
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:43111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:42013,127.27.225.253:43287,127.27.225.252:42185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:13.008744 18875 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 21:11:13.009198 18875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:13.009294 18875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:13.009399 18875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:13.014139 18875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:13.014235 18875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:13.014273 18875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:13.014318 18875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:13.014353 18875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:11:13.020628 18875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:43111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:42013,127.27.225.253:43287,127.27.225.252:42185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:42013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:13.022058 18875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:13.023175 18875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:11:13.032812 18875 server_base.cc:1034] running on GCE node
W20260806 21:11:13.032755 18881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:13.032755 18880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:13.032761 18883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:13.033864 18875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:13.035036 18875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:13.036216 18875 hybrid_clock.cc:648] HybridClock initialized: now 1786050673036204 us; error 29 us; skew 500 ppm
Aug 06 21:11:13 dist-test-slave-69rg krb5kdc[18862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050673, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:13.039518 18875 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:11:13.040513 18875 webserver.cc:460] Webserver started at http://127.27.225.254:46813/ using document root <none> and password file <none>
I20260806 21:11:13.041119 18875 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:13.041231 18875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:13.041477 18875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:13.043756 18875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "23bbb0bb53b4488d836024f02e1d8dae"
format_stamp: "Formatted at 2026-08-06 21:11:13 on dist-test-slave-69rg"
server_key: "3c51856ac1576c71b349b8099482bb81"
server_key_iv: "bbbdb701017cbf6da85d0d87784c9d3d"
server_key_version: "encryptionkey@0"
I20260806 21:11:13.044371 18875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "23bbb0bb53b4488d836024f02e1d8dae"
format_stamp: "Formatted at 2026-08-06 21:11:13 on dist-test-slave-69rg"
server_key: "3c51856ac1576c71b349b8099482bb81"
server_key_iv: "bbbdb701017cbf6da85d0d87784c9d3d"
server_key_version: "encryptionkey@0"
I20260806 21:11:13.048866 18875 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:11:13.051759 18890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:13.053083 18875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 21:11:13.053267 18875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "23bbb0bb53b4488d836024f02e1d8dae"
format_stamp: "Formatted at 2026-08-06 21:11:13 on dist-test-slave-69rg"
server_key: "3c51856ac1576c71b349b8099482bb81"
server_key_iv: "bbbdb701017cbf6da85d0d87784c9d3d"
server_key_version: "encryptionkey@0"
I20260806 21:11:13.053411 18875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:13.076006 18875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:13.080420 18875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:13.080713 18875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:13.089126 18875 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:42013
I20260806 21:11:13.089480 18943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:42013 every 8 connection(s)
I20260806 21:11:13.090262 18875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:11:13.093475 18944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:13.095286 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 18875
I20260806 21:11:13.095458 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:11:13.095817 28551 external_mini_cluster.cc:1428] Setting key 167baf40eb7d465b99639223bea891ab
I20260806 21:11:13.099538 18944 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42013 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.27.225.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.253@KRBTEST.COM" created.
Aug 06 21:11:13 dist-test-slave-69rg krb5kdc[18862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050673, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:11:13.119240 18948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.103933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42826 (local address 127.27.225.254:42013)
0806 21:11:13.104308 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.104321 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.104371 (+ 50us) server_negotiation.cc:407] Connection header received
0806 21:11:13.104950 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.104957 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.105255 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.105491 (+ 236us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.106521 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.107739 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.109159 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.109718 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.109978 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:13.113361 (+ 3383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:13.113372 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:13.113393 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:13.113428 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:13.115459 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.116115 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.116120 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.116122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.116244 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.116804 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.116808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.116809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.117328 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:13.117475 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:13.118052 (+ 577us) server_negotiation.cc:299] Negotiation successful
0806 21:11:13.118335 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":105,"threads_started":1}
I20260806 21:11:13.119256 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.102925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:42013 (local address 127.0.0.1:42826)
0806 21:11:13.103503 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.103559 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:13.104587 (+ 1028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:13.105519 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:13.105524 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:13.106011 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:13.106334 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:13.106350 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.107973 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.107980 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:13.108993 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:13.109005 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.110245 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.110251 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:13.110410 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:13.111171 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:13.111190 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:13.113160 (+ 1970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:13.115589 (+ 2429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:13.115602 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:13.115619 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:13.115976 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:13.116448 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:13.116456 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:13.116458 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:13.116648 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:13.117490 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:13.117499 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:13.117707 (+ 208us) client_negotiation.cc:769] Sending connection context
0806 21:11:13.117953 (+ 246us) client_negotiation.cc:240] Negotiation successful
0806 21:11:13.118490 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":112,"thread_start_us":187,"threads_started":1}
I20260806 21:11:13.122430 18944 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:11:13.123274 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.122979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:43287 (local address 127.0.0.1:50006)
0806 21:11:13.123115 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.123202 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.253:43287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 21:11:13.123571 18892 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.225.253:43287: connect: Connection refused (error 111)
I20260806 21:11:13.124488 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.124280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:43287 (local address 127.0.0.1:50010)
0806 21:11:13.124386 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.124432 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.253:43287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 21:11:13.124760 18944 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.253:43287: Network error: Client connection negotiation failed: client connection to 127.27.225.253:43287: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.253.keytab.
Entry for principal kudu/127.27.225.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.253.keytab.
I20260806 21:11:13.130128 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.253:43287
--webserver_interface=127.27.225.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.253.keytab
--principal=kudu/127.27.225.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:43111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:42013,127.27.225.253:43287,127.27.225.252:42185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:11:13.174039 18944 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 } attempt: 1
I20260806 21:11:13.175053 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.174736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:43287 (local address 127.0.0.1:50018)
0806 21:11:13.174908 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.174972 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.253:43287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 21:11:13.175861 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.175653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:43287 (local address 127.0.0.1:50030)
0806 21:11:13.175754 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.175806 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.253:43287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 21:11:13.176015 18944 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.253:43287: Network error: Client connection negotiation failed: client connection to 127.27.225.253:43287: connect: Connection refused (error 111)
I20260806 21:11:13.244257 18944 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 } attempt: 2
I20260806 21:11:13.245350 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.244932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:43287 (local address 127.0.0.1:50042)
0806 21:11:13.245188 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.245261 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.253:43287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260806 21:11:13.246450 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.246148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:43287 (local address 127.0.0.1:50044)
0806 21:11:13.246324 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.246379 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.253:43287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260806 21:11:13.246856 18944 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.253:43287: Network error: Client connection negotiation failed: client connection to 127.27.225.253:43287: connect: Connection refused (error 111)
W20260806 21:11:13.261802 18951 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 21:11:13.262230 18951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:13.262256 18951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:13.262298 18951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:13.267503 18951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:13.267647 18951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:13.267689 18951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:13.267724 18951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:13.267741 18951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:11:13.275264 18951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:43111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:42013,127.27.225.253:43287,127.27.225.252:42185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.253.keytab
--principal=kudu/127.27.225.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.253:43287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.225.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:13.276533 18951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:13.277727 18951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:13.288746 18960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:13.288757 18956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:13.289068 18951 server_base.cc:1034] running on GCE node
W20260806 21:11:13.288787 18958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:13.289860 18951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:13.291106 18951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:13.292313 18951 hybrid_clock.cc:648] HybridClock initialized: now 1786050673292286 us; error 41 us; skew 500 ppm
Aug 06 21:11:13 dist-test-slave-69rg krb5kdc[18862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050673, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:13.295413 18951 init.cc:377] Logged in from keytab as kudu/127.27.225.253@KRBTEST.COM (short username kudu)
I20260806 21:11:13.296555 18951 webserver.cc:460] Webserver started at http://127.27.225.253:32773/ using document root <none> and password file <none>
I20260806 21:11:13.297184 18951 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:13.297289 18951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:13.297538 18951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:13.299860 18951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-1/data/instance:
uuid: "247dedc1aff84f0587ee62e2097743e5"
format_stamp: "Formatted at 2026-08-06 21:11:13 on dist-test-slave-69rg"
server_key: "00337b4d147dc8966a970fca34989714"
server_key_iv: "ae901b4bf9da3a7c20bc6220201d08a0"
server_key_version: "encryptionkey@0"
I20260806 21:11:13.300510 18951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-1/wal/instance:
uuid: "247dedc1aff84f0587ee62e2097743e5"
format_stamp: "Formatted at 2026-08-06 21:11:13 on dist-test-slave-69rg"
server_key: "00337b4d147dc8966a970fca34989714"
server_key_iv: "ae901b4bf9da3a7c20bc6220201d08a0"
server_key_version: "encryptionkey@0"
I20260806 21:11:13.304704 18951 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 21:11:13.307526 18968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:13.308969 18951 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 21:11:13.309157 18951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-1/wal
uuid: "247dedc1aff84f0587ee62e2097743e5"
format_stamp: "Formatted at 2026-08-06 21:11:13 on dist-test-slave-69rg"
server_key: "00337b4d147dc8966a970fca34989714"
server_key_iv: "ae901b4bf9da3a7c20bc6220201d08a0"
server_key_version: "encryptionkey@0"
I20260806 21:11:13.309319 18951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:13.330192 18951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:13.334167 18951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:13.334444 18951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:13.338111 18944 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 } attempt: 3
I20260806 21:11:13.338968 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.338668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:43287 (local address 127.0.0.1:50052)
0806 21:11:13.338795 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.338866 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.253:43287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 21:11:13.339922 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.339581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:43287 (local address 127.0.0.1:50060)
0806 21:11:13.339742 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.339821 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.253:43287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 21:11:13.340094 18944 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.253:43287: Network error: Client connection negotiation failed: client connection to 127.27.225.253:43287: connect: Connection refused (error 111)
I20260806 21:11:13.343117 18951 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.253:43287
I20260806 21:11:13.343442 19021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.253:43287 every 8 connection(s)
I20260806 21:11:13.344287 18951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-1/data/info.pb
I20260806 21:11:13.347477 19022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:13.349092 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 18951
I20260806 21:11:13.349220 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-1/wal/instance
I20260806 21:11:13.349567 28551 external_mini_cluster.cc:1428] Setting key 2a1951673e57e2bc40bd25e01eb2bd3e
I20260806 21:11:13.352005 19022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42013 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.27.225.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.252@KRBTEST.COM" created.
Aug 06 21:11:13 dist-test-slave-69rg krb5kdc[18862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050673, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.253@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:11:13.367786 18948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.354272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42828 (local address 127.27.225.254:42013)
0806 21:11:13.354466 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.354470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.355727 (+ 1257us) server_negotiation.cc:407] Connection header received
0806 21:11:13.356844 (+ 1117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.356848 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.356913 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.357027 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.358486 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.359025 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.360245 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.360600 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.360725 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:13.363770 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:13.363774 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:13.363776 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:13.363805 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:13.365371 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.366058 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.366062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.366063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.366158 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.366545 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.366548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.366550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.366757 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:13.366870 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:13.367457 (+ 587us) server_negotiation.cc:299] Negotiation successful
0806 21:11:13.367603 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 21:11:13.368651 19025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.354888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:42013 (local address 127.0.0.1:42828)
0806 21:11:13.355541 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.355591 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:13.356617 (+ 1026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:13.357266 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:13.357273 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:13.357689 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:13.358335 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:13.358352 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.359168 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.359172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:13.360106 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:13.360117 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.360791 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.360796 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:13.360978 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:13.361647 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:13.361669 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:13.363523 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:13.365521 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:13.365529 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:13.365547 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:13.365931 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:13.366268 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:13.366271 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:13.366273 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:13.366435 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:13.366869 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:13.366878 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:13.367181 (+ 303us) client_negotiation.cc:769] Sending connection context
0806 21:11:13.367383 (+ 202us) client_negotiation.cc:240] Negotiation successful
0806 21:11:13.367757 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":201,"threads_started":1}
I20260806 21:11:13.370373 19022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 21:11:13 dist-test-slave-69rg krb5kdc[18862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050673, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.253@KRBTEST.COM for kudu/127.27.225.253@KRBTEST.COM
I20260806 21:11:13.379848 19025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.371042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:43287 (local address 127.0.0.1:50074)
0806 21:11:13.371161 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.371175 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:13.371256 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:13.371840 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:13.371843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:13.372033 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:13.372162 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:13.372168 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.373143 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.373147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:13.373678 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:13.373686 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.374354 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.374358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:13.374441 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:13.374964 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:13.374977 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:13.376204 (+ 1227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:13.378269 (+ 2065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:13.378273 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:13.378275 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:13.378467 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:13.378774 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:13.378777 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:13.378779 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:13.378857 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:13.379320 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:13.379323 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:13.379409 (+ 86us) client_negotiation.cc:769] Sending connection context
0806 21:11:13.379524 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 21:11:13.379664 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 21:11:13.379904 19027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.371145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50074 (local address 127.27.225.253:43287)
0806 21:11:13.371481 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.371490 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.371509 (+ 19us) server_negotiation.cc:407] Connection header received
0806 21:11:13.371544 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.371548 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.371710 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.371877 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.372274 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.373020 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.373795 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.374231 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.374344 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:13.376353 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:13.376363 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:13.376376 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:13.376396 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:13.378148 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.378586 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.378592 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.378594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.378677 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.378950 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.378953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.378955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.379211 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:13.379350 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:13.379634 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 21:11:13.379766 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":10,"server-negotiator.queue_time_us":251,"thread_start_us":130,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.252.keytab.
Entry for principal kudu/127.27.225.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.252.keytab.
I20260806 21:11:13.381721 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.252:42185
--webserver_interface=127.27.225.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.252.keytab
--principal=kudu/127.27.225.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:43111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:42013,127.27.225.253:43287,127.27.225.252:42185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:11:13.381956 19022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:11:13.382717 19025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.382442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:42185 (local address 127.0.0.1:37222)
0806 21:11:13.382565 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.382642 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:42185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 21:11:13.382998 18971 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.225.252:42185: connect: Connection refused (error 111)
I20260806 21:11:13.384009 19025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.383747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:42185 (local address 127.0.0.1:37224)
0806 21:11:13.383870 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.383931 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:42185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 21:11:13.384240 19022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.252:42185: Network error: Client connection negotiation failed: client connection to 127.27.225.252:42185: connect: Connection refused (error 111)
I20260806 21:11:13.433554 19022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } attempt: 1
I20260806 21:11:13.434597 19025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.434272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:42185 (local address 127.0.0.1:37234)
0806 21:11:13.434424 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.434495 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:42185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 21:11:13.435503 19025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.435211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:42185 (local address 127.0.0.1:37248)
0806 21:11:13.435383 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.435439 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:42185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260806 21:11:13.435683 19022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.252:42185: Network error: Client connection negotiation failed: client connection to 127.27.225.252:42185: connect: Connection refused (error 111)
I20260806 21:11:13.483309 18944 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 } attempt: 4
Aug 06 21:11:13 dist-test-slave-69rg krb5kdc[18862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050673, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for kudu/127.27.225.253@KRBTEST.COM
I20260806 21:11:13.493006 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.483934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:43287 (local address 127.0.0.1:50078)
0806 21:11:13.484207 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.484221 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:13.484348 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:13.484638 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:13.484642 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:13.484840 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:13.484958 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:13.484964 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.486142 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.486148 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:13.487066 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:13.487076 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.487572 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.487575 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:13.487682 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:13.488217 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:13.488229 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:13.489691 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:13.491476 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:13.491479 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:13.491481 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:13.491684 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:13.491978 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:13.491982 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:13.491984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:13.492068 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:13.492458 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:13.492461 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:13.492566 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 21:11:13.492676 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 21:11:13.492824 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195}
I20260806 21:11:13.493006 19027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.483991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50078 (local address 127.27.225.253:43287)
0806 21:11:13.484144 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.484149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.484297 (+ 148us) server_negotiation.cc:407] Connection header received
0806 21:11:13.484471 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.484473 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.484519 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.484609 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.485080 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.486032 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.487198 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.487451 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.487565 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:13.489867 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:13.489872 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:13.489875 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:13.489908 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:13.491342 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.491793 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.491797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.491799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.491880 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.492166 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.492169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.492171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.492361 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:13.492466 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:13.492727 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 21:11:13.492853 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 21:11:13.493887 18944 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:11:13.494632 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.494406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:42185 (local address 127.0.0.1:37254)
0806 21:11:13.494526 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.494574 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:42185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 21:11:13.495342 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.495088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:42185 (local address 127.0.0.1:37268)
0806 21:11:13.495233 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.495278 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:42185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260806 21:11:13.495482 18944 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.252:42185: Network error: Client connection negotiation failed: client connection to 127.27.225.252:42185: connect: Connection refused (error 111)
W20260806 21:11:13.500170 19029 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 21:11:13.500484 19029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:13.500540 19029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:13.500613 19029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 21:11:13.503911 19022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } attempt: 2
I20260806 21:11:13.504853 19025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.504540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:42185 (local address 127.0.0.1:37278)
0806 21:11:13.504707 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.504773 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:42185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260806 21:11:13.505120 19029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:13.505196 19029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:13.505241 19029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:13.505277 19029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:13.505326 19029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:11:13.505808 19025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.505547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:42185 (local address 127.0.0.1:37284)
0806 21:11:13.505698 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.505752 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:42185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260806 21:11:13.506160 19022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.252:42185: Network error: Client connection negotiation failed: client connection to 127.27.225.252:42185: connect: Connection refused (error 111)
I20260806 21:11:13.511565 19029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:43111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:42013,127.27.225.253:43287,127.27.225.252:42185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.252.keytab
--principal=kudu/127.27.225.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.252:42185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.225.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:13.512729 19029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:13.513935 19029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:13.523000 19034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:13.522995 19035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:13.522995 19037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:13.523373 19029 server_base.cc:1034] running on GCE node
I20260806 21:11:13.524003 19029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:13.525117 19029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:13.526284 19029 hybrid_clock.cc:648] HybridClock initialized: now 1786050673526250 us; error 47 us; skew 500 ppm
Aug 06 21:11:13 dist-test-slave-69rg krb5kdc[18862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050673, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:13.529174 19029 init.cc:377] Logged in from keytab as kudu/127.27.225.252@KRBTEST.COM (short username kudu)
I20260806 21:11:13.530233 19029 webserver.cc:460] Webserver started at http://127.27.225.252:36911/ using document root <none> and password file <none>
I20260806 21:11:13.530790 19029 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:13.530855 19029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:13.531031 19029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:13.533207 19029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-2/data/instance:
uuid: "95a627c752f54dc4b58866e3d22ddaa8"
format_stamp: "Formatted at 2026-08-06 21:11:13 on dist-test-slave-69rg"
server_key: "8f228fc12306110d3f736bb6e962e0a6"
server_key_iv: "b1a7a69eae0bb012df7499b300af598f"
server_key_version: "encryptionkey@0"
I20260806 21:11:13.533871 19029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-2/wal/instance:
uuid: "95a627c752f54dc4b58866e3d22ddaa8"
format_stamp: "Formatted at 2026-08-06 21:11:13 on dist-test-slave-69rg"
server_key: "8f228fc12306110d3f736bb6e962e0a6"
server_key_iv: "b1a7a69eae0bb012df7499b300af598f"
server_key_version: "encryptionkey@0"
I20260806 21:11:13.537922 19029 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 21:11:13.540539 19045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:13.542167 19029 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 21:11:13.542361 19029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-2/wal
uuid: "95a627c752f54dc4b58866e3d22ddaa8"
format_stamp: "Formatted at 2026-08-06 21:11:13 on dist-test-slave-69rg"
server_key: "8f228fc12306110d3f736bb6e962e0a6"
server_key_iv: "b1a7a69eae0bb012df7499b300af598f"
server_key_version: "encryptionkey@0"
I20260806 21:11:13.542497 19029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:13.554652 18944 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } attempt: 1
I20260806 21:11:13.555512 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.555187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:42185 (local address 127.0.0.1:37288)
0806 21:11:13.555349 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.555421 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:42185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 21:11:13.556258 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.556046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:42185 (local address 127.0.0.1:37300)
0806 21:11:13.556152 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.556204 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:42185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 21:11:13.556404 18944 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.252:42185: Network error: Client connection negotiation failed: client connection to 127.27.225.252:42185: connect: Connection refused (error 111)
I20260806 21:11:13.560488 19029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:13.563951 19029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:13.564208 19029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:13.572345 19029 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.252:42185
I20260806 21:11:13.572670 19099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.252:42185 every 8 connection(s)
I20260806 21:11:13.573479 19029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-2/data/info.pb
I20260806 21:11:13.576318 19100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:13.580202 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 19029
I20260806 21:11:13.580345 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050378084738-28551-0/minicluster-data/master-2/wal/instance
I20260806 21:11:13.580042 19100 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42013 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:11:13.580632 28551 external_mini_cluster.cc:1428] Setting key a508a5eb092c3b271559419cc348ca8c
Aug 06 21:11:13 dist-test-slave-69rg krb5kdc[18862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050673, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.252@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
Aug 06 21:11:13 dist-test-slave-69rg krb5kdc[18862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.252@KRBTEST.COM
I20260806 21:11:13.594074 18948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.582328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42844 (local address 127.27.225.254:42013)
0806 21:11:13.582457 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.582461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.582950 (+ 489us) server_negotiation.cc:407] Connection header received
0806 21:11:13.583814 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.583817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.583869 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.583945 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.584918 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.585433 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.586552 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.586775 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.586886 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:13.589852 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:13.589857 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:13.589860 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:13.589896 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:13.591579 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.592295 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.592298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.592299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.592385 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.592830 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.592833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.592835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.593055 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:13.593177 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:13.593692 (+ 515us) server_negotiation.cc:299] Negotiation successful
0806 21:11:13.593895 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 21:11:13.594928 19103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.582215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:42013 (local address 127.0.0.1:42844)
0806 21:11:13.582783 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.582836 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:13.583591 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:13.584017 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:13.584022 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:13.584480 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:13.584756 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:13.584774 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.585545 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.585548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:13.586427 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:13.586436 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.586937 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.586941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:13.587133 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:13.587756 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:13.587781 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:13.589596 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:13.591694 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:13.591703 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:13.591713 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:13.592171 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:13.592516 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:13.592520 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:13.592522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:13.592687 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:13.593180 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:13.593189 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:13.593474 (+ 285us) client_negotiation.cc:769] Sending connection context
0806 21:11:13.593648 (+ 174us) client_negotiation.cc:240] Negotiation successful
0806 21:11:13.594031 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":55,"thread_start_us":148,"threads_started":1}
I20260806 21:11:13.594933 19104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.582215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37306 (local address 127.27.225.252:42185)
0806 21:11:13.582733 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.582751 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.582789 (+ 38us) server_negotiation.cc:407] Connection header received
0806 21:11:13.583591 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.583614 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.583848 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.584030 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.584780 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.585777 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.586677 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.587101 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.587220 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:13.590323 (+ 3103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:13.590329 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:13.590339 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:13.590361 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:13.592209 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.592702 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.592708 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.592710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.592793 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.593079 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.593084 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.593086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.593458 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:13.593668 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:13.593936 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 21:11:13.594084 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":402,"spinlock_wait_cycles":48512,"thread_start_us":175,"threads_started":1}
I20260806 21:11:13.596189 19100 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:11:13.597352 19022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } attempt: 3
Aug 06 21:11:13 dist-test-slave-69rg krb5kdc[18862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050673, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.252@KRBTEST.COM for kudu/127.27.225.253@KRBTEST.COM
Aug 06 21:11:13 dist-test-slave-69rg krb5kdc[18862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050673, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.253@KRBTEST.COM for kudu/127.27.225.252@KRBTEST.COM
Aug 06 21:11:13 dist-test-slave-69rg krb5kdc[18862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:11:13.606200 19027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.596884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50092 (local address 127.27.225.253:43287)
0806 21:11:13.597032 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.597039 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.597057 (+ 18us) server_negotiation.cc:407] Connection header received
0806 21:11:13.597105 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.597108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.597151 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.597238 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.597884 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.598570 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.599703 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.599974 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.600086 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:13.602385 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:13.602391 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:13.602394 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:13.602422 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:13.604058 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.604620 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.604624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.604625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.604709 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.605061 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.605066 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.605068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.605408 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:13.605489 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:13.605871 (+ 382us) server_negotiation.cc:299] Negotiation successful
0806 21:11:13.605999 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":59}
I20260806 21:11:13.606765 19103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.596739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:43287 (local address 127.0.0.1:50092)
0806 21:11:13.596900 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.596931 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:13.597011 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:13.597367 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:13.597370 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:13.597572 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:13.597722 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:13.597730 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.598769 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.598772 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:13.599590 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:13.599597 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.600088 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.600091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:13.600191 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:13.600666 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:13.600680 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:13.602254 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:13.604171 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:13.604174 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:13.604177 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:13.604510 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:13.604819 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:13.604823 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:13.604826 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:13.604922 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:13.605513 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:13.605518 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:13.605625 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 21:11:13.606432 (+ 807us) client_negotiation.cc:240] Negotiation successful
0806 21:11:13.606577 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260806 21:11:13.607538 19100 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:11:13.607789 19025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.597745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:42185 (local address 127.0.0.1:37318)
0806 21:11:13.597879 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.597893 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:13.597965 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:13.598323 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:13.598329 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:13.598503 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:13.598612 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:13.598622 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.599319 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.599323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:13.600140 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:13.600150 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.601000 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.601004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:13.601082 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:13.601806 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:13.601849 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:13.604062 (+ 2213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:13.606205 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:13.606210 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:13.606212 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:13.606462 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:13.606748 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:13.606751 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:13.606753 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:13.606858 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:13.607273 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:13.607277 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:13.607372 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 21:11:13.607475 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 21:11:13.607612 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":1}
I20260806 21:11:13.607859 18948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.597288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42846 (local address 127.27.225.254:42013)
0806 21:11:13.597575 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.597584 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.597595 (+ 11us) server_negotiation.cc:407] Connection header received
0806 21:11:13.597624 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.597626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.597658 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.597754 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.598640 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.599424 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.600490 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.600831 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.600941 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:13.604251 (+ 3310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:13.604257 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:13.604261 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:13.604289 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:13.606145 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.606616 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.606620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.606621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.606708 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.607003 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.607006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.607008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.607253 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:13.607502 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:13.607587 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 21:11:13.607709 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202}
I20260806 21:11:13.608302 19104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.597915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37318 (local address 127.27.225.252:42185)
0806 21:11:13.598097 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.598101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.598121 (+ 20us) server_negotiation.cc:407] Connection header received
0806 21:11:13.598153 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.598156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.598200 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.598274 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.598708 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.599198 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.600270 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.600847 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.601627 (+ 780us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:13.604209 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:13.604212 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:13.604215 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:13.604244 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:13.606088 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.606548 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.606551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.606553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.606639 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.606975 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.606980 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.606981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.607182 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:13.607273 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:13.608064 (+ 791us) server_negotiation.cc:299] Negotiation successful
0806 21:11:13.608174 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260806 21:11:13.611660 19022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 247dedc1aff84f0587ee62e2097743e5: Bootstrap starting.
Aug 06 21:11:13 dist-test-slave-69rg krb5kdc[18862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050673, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.252@KRBTEST.COM for kudu/127.27.225.252@KRBTEST.COM
I20260806 21:11:13.614760 19022 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 247dedc1aff84f0587ee62e2097743e5: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:13.615746 19022 log.cc:824] T 00000000000000000000000000000000 P 247dedc1aff84f0587ee62e2097743e5: Log is configured to *not* fsync() on all Append() calls
Aug 06 21:11:13 dist-test-slave-69rg krb5kdc[18862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.253@KRBTEST.COM
I20260806 21:11:13.617192 19103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.608045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:42185 (local address 127.0.0.1:37326)
0806 21:11:13.608590 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.608605 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:13.608669 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:13.609181 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:13.609184 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:13.609354 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:13.609445 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:13.609455 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.610424 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.610427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:13.611155 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:13.611165 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.611759 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.611764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:13.611839 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:13.612317 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:13.612331 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:13.613930 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:13.615561 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:13.615567 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:13.615569 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:13.615763 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:13.616055 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:13.616060 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:13.616062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:13.616143 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:13.616718 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:13.616720 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:13.616784 (+ 64us) client_negotiation.cc:769] Sending connection context
0806 21:11:13.616888 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 21:11:13.617023 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 21:11:13.617203 19104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.608509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37326 (local address 127.27.225.252:42185)
0806 21:11:13.608876 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.608882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.608897 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:11:13.608934 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.608939 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.608981 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.609050 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.609618 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.610310 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.611288 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.611629 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.611728 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:13.614050 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:13.614053 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:13.614055 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:13.614075 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:13.615454 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.615855 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.615858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.615860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.615967 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.616257 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.616259 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.616261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.616572 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:13.616679 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:13.616930 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 21:11:13.617079 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":300}
I20260806 21:11:13.618397 19022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 247dedc1aff84f0587ee62e2097743e5: No bootstrap required, opened a new log
I20260806 21:11:13.620739 19027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.609036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50106 (local address 127.27.225.253:43287)
0806 21:11:13.609130 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.609134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.610055 (+ 921us) server_negotiation.cc:407] Connection header received
0806 21:11:13.610255 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.610258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.610298 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.610387 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.610973 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.611749 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.612761 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.613075 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.613244 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:13.616964 (+ 3720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:13.616968 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:13.616970 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:13.616994 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:13.618905 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.619398 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.619404 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.619406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.619517 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.619832 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.619835 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.619837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.620067 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:13.620253 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:13.620470 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 21:11:13.620588 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 21:11:13.621306 19100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95a627c752f54dc4b58866e3d22ddaa8: Bootstrap starting.
I20260806 21:11:13.622370 19022 raft_consensus.cc:348] T 00000000000000000000000000000000 P 247dedc1aff84f0587ee62e2097743e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23bbb0bb53b4488d836024f02e1d8dae" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42013 } } peers { permanent_uuid: "247dedc1aff84f0587ee62e2097743e5" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 } } peers { permanent_uuid: "95a627c752f54dc4b58866e3d22ddaa8" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } }
I20260806 21:11:13.622639 19022 raft_consensus.cc:729] T 00000000000000000000000000000000 P 247dedc1aff84f0587ee62e2097743e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 247dedc1aff84f0587ee62e2097743e5, State: Initialized, Role: FOLLOWER
I20260806 21:11:13.622994 19022 consensus_queue.cc:260] T 00000000000000000000000000000000 P 247dedc1aff84f0587ee62e2097743e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23bbb0bb53b4488d836024f02e1d8dae" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42013 } } peers { permanent_uuid: "247dedc1aff84f0587ee62e2097743e5" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 } } peers { permanent_uuid: "95a627c752f54dc4b58866e3d22ddaa8" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } }
I20260806 21:11:13.623498 19106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 247dedc1aff84f0587ee62e2097743e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23bbb0bb53b4488d836024f02e1d8dae" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42013 } } peers { permanent_uuid: "247dedc1aff84f0587ee62e2097743e5" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 } } peers { permanent_uuid: "95a627c752f54dc4b58866e3d22ddaa8" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } } }
I20260806 21:11:13.623581 18944 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } attempt: 2
I20260806 21:11:13.623651 19106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 247dedc1aff84f0587ee62e2097743e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:11:13.624207 19100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95a627c752f54dc4b58866e3d22ddaa8: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:13.624795 19022 sys_catalog.cc:564] T 00000000000000000000000000000000 P 247dedc1aff84f0587ee62e2097743e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:13.625131 19100 log.cc:824] T 00000000000000000000000000000000 P 95a627c752f54dc4b58866e3d22ddaa8: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:13.625792 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:11:13.627382 18917 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "23bbb0bb53b4488d836024f02e1d8dae"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42846
Aug 06 21:11:13 dist-test-slave-69rg krb5kdc[18862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050673, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for kudu/127.27.225.252@KRBTEST.COM
W20260806 21:11:13.630544 19118 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 247dedc1aff84f0587ee62e2097743e5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:11:13.630620 19118 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:11:13.632647 19100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95a627c752f54dc4b58866e3d22ddaa8: No bootstrap required, opened a new log
I20260806 21:11:13.634256 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.624071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:42185 (local address 127.0.0.1:37328)
0806 21:11:13.624363 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:13.624379 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:13.624479 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:13.624994 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:13.624998 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:13.625205 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:13.625335 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:13.625343 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.626475 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.626481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:13.627295 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:13.627306 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.627886 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.627889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:13.627968 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:13.628556 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:13.628571 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:13.630204 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:13.632314 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:13.632318 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:13.632320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:13.632600 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:13.632948 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:13.632951 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:13.632953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:13.633066 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:13.633616 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:13.633620 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:13.633748 (+ 128us) client_negotiation.cc:769] Sending connection context
0806 21:11:13.633911 (+ 163us) client_negotiation.cc:240] Negotiation successful
0806 21:11:13.634069 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199}
I20260806 21:11:13.630999 18951 master_runner.cc:186] Error getting master registration for 127.27.225.254:42013: OK, instance_id { permanent_uuid: "23bbb0bb53b4488d836024f02e1d8dae" instance_seqno: 1786050673086844 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 21:11:13.634701 18951 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 21:11:13.634840 19104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.624627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37328 (local address 127.27.225.252:42185)
0806 21:11:13.624738 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.624742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.624755 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:11:13.624799 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.624802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.624844 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.625456 (+ 612us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.625567 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.626329 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.627423 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.627754 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.627862 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:13.630368 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:13.630372 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:13.630375 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:13.630402 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:13.632166 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.632722 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.632726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.632728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.632821 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.633201 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.633204 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.633206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.633457 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:13.633552 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:13.634556 (+ 1004us) server_negotiation.cc:299] Negotiation successful
0806 21:11:13.634686 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 21:11:13.636547 19100 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95a627c752f54dc4b58866e3d22ddaa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23bbb0bb53b4488d836024f02e1d8dae" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42013 } } peers { permanent_uuid: "247dedc1aff84f0587ee62e2097743e5" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 } } peers { permanent_uuid: "95a627c752f54dc4b58866e3d22ddaa8" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } }
I20260806 21:11:13.636813 19100 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95a627c752f54dc4b58866e3d22ddaa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95a627c752f54dc4b58866e3d22ddaa8, State: Initialized, Role: FOLLOWER
I20260806 21:11:13.637140 19100 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95a627c752f54dc4b58866e3d22ddaa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23bbb0bb53b4488d836024f02e1d8dae" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42013 } } peers { permanent_uuid: "247dedc1aff84f0587ee62e2097743e5" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 } } peers { permanent_uuid: "95a627c752f54dc4b58866e3d22ddaa8" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } }
I20260806 21:11:13.637632 19122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95a627c752f54dc4b58866e3d22ddaa8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23bbb0bb53b4488d836024f02e1d8dae" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42013 } } peers { permanent_uuid: "247dedc1aff84f0587ee62e2097743e5" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 } } peers { permanent_uuid: "95a627c752f54dc4b58866e3d22ddaa8" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } } }
I20260806 21:11:13.637809 19122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95a627c752f54dc4b58866e3d22ddaa8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:11:13.638221 18944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae: Bootstrap starting.
I20260806 21:11:13.638581 19100 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95a627c752f54dc4b58866e3d22ddaa8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:13.641136 18944 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:13.642119 18944 log.cc:824] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae: Log is configured to *not* fsync() on all Append() calls
W20260806 21:11:13.644032 19133 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 95a627c752f54dc4b58866e3d22ddaa8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:11:13.644181 19133 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:11:13.644501 18944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae: No bootstrap required, opened a new log
I20260806 21:11:13.642148 19029 master_runner.cc:186] Error getting master registration for 127.27.225.254:42013: OK, instance_id { permanent_uuid: "23bbb0bb53b4488d836024f02e1d8dae" instance_seqno: 1786050673086844 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 21:11:13.645689 19029 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 21:11:13.647851 18944 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23bbb0bb53b4488d836024f02e1d8dae" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42013 } } peers { permanent_uuid: "247dedc1aff84f0587ee62e2097743e5" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 } } peers { permanent_uuid: "95a627c752f54dc4b58866e3d22ddaa8" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } }
I20260806 21:11:13.648084 18944 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23bbb0bb53b4488d836024f02e1d8dae, State: Initialized, Role: FOLLOWER
I20260806 21:11:13.648416 18944 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23bbb0bb53b4488d836024f02e1d8dae" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42013 } } peers { permanent_uuid: "247dedc1aff84f0587ee62e2097743e5" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 } } peers { permanent_uuid: "95a627c752f54dc4b58866e3d22ddaa8" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } }
I20260806 21:11:13.649011 19137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23bbb0bb53b4488d836024f02e1d8dae" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42013 } } peers { permanent_uuid: "247dedc1aff84f0587ee62e2097743e5" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 } } peers { permanent_uuid: "95a627c752f54dc4b58866e3d22ddaa8" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } } }
I20260806 21:11:13.649303 19137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:11:13.649844 18944 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:13.649888 18917 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 21:11:13.650002 18917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:13.651371 18917 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23bbb0bb53b4488d836024f02e1d8dae" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42013 } } peers { permanent_uuid: "247dedc1aff84f0587ee62e2097743e5" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 } } peers { permanent_uuid: "95a627c752f54dc4b58866e3d22ddaa8" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } }
I20260806 21:11:13.652429 18917 leader_election.cc:288] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae [CANDIDATE]: Term 1 election: Requested vote from peers 247dedc1aff84f0587ee62e2097743e5 (127.27.225.253:43287), 95a627c752f54dc4b58866e3d22ddaa8 (127.27.225.252:42185)
I20260806 21:11:13.653179 18996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "23bbb0bb53b4488d836024f02e1d8dae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "247dedc1aff84f0587ee62e2097743e5"
I20260806 21:11:13.653506 18996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 247dedc1aff84f0587ee62e2097743e5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:13.654896 18996 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 247dedc1aff84f0587ee62e2097743e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23bbb0bb53b4488d836024f02e1d8dae in term 1.
I20260806 21:11:13.654884 19072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "23bbb0bb53b4488d836024f02e1d8dae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "95a627c752f54dc4b58866e3d22ddaa8"
I20260806 21:11:13.655145 19072 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95a627c752f54dc4b58866e3d22ddaa8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:13.656253 18892 leader_election.cc:302] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23bbb0bb53b4488d836024f02e1d8dae, 247dedc1aff84f0587ee62e2097743e5; no voters:
I20260806 21:11:13.656454 19072 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 95a627c752f54dc4b58866e3d22ddaa8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23bbb0bb53b4488d836024f02e1d8dae in term 1.
I20260806 21:11:13.656599 19137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:13.656873 19137 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae [term 1 LEADER]: Becoming Leader. State: Replica: 23bbb0bb53b4488d836024f02e1d8dae, State: Running, Role: LEADER
I20260806 21:11:13.657325 19137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23bbb0bb53b4488d836024f02e1d8dae" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42013 } } peers { permanent_uuid: "247dedc1aff84f0587ee62e2097743e5" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 } } peers { permanent_uuid: "95a627c752f54dc4b58866e3d22ddaa8" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } }
I20260806 21:11:13.659368 19137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23bbb0bb53b4488d836024f02e1d8dae. Latest consensus state: current_term: 1 leader_uuid: "23bbb0bb53b4488d836024f02e1d8dae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23bbb0bb53b4488d836024f02e1d8dae" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42013 } } peers { permanent_uuid: "247dedc1aff84f0587ee62e2097743e5" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 } } peers { permanent_uuid: "95a627c752f54dc4b58866e3d22ddaa8" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } } }
I20260806 21:11:13.659559 19137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae [sys.catalog]: This master's current role is: LEADER
W20260806 21:11:13.660347 19148 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:11:13.660427 19148 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:11:13.661777 18875 master_runner.cc:263] No leader master found from master 23bbb0bb53b4488d836024f02e1d8dae
I20260806 21:11:13.661890 18875 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 21:11:13.672250 18948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.654896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42860 (local address 127.27.225.254:42013)
0806 21:11:13.656761 (+ 1865us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.656766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.662208 (+ 5442us) server_negotiation.cc:407] Connection header received
0806 21:11:13.662411 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.662415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.662464 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.662517 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.663690 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.664510 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.665858 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.666212 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.666294 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:13.668352 (+ 2058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:13.668357 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:13.668360 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:13.668394 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:13.670257 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.670815 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.670818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.670820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.670910 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.671204 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.671207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.671209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.671441 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:13.671556 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:13.671935 (+ 379us) server_negotiation.cc:299] Negotiation successful
0806 21:11:13.672071 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1777}
I20260806 21:11:13.678872 19104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.655785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37338 (local address 127.27.225.252:42185)
0806 21:11:13.656853 (+ 1068us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.656857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.666134 (+ 9277us) server_negotiation.cc:407] Connection header received
0806 21:11:13.666329 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.666332 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.666387 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.666439 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.667482 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.668270 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.669431 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.669780 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.669917 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:13.675096 (+ 5179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:13.675101 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:13.675103 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:13.675134 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:13.676900 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.677406 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.677410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.677412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.677499 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.677832 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.677836 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.677838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.678084 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:13.678191 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:13.678565 (+ 374us) server_negotiation.cc:299] Negotiation successful
0806 21:11:13.678701 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":997}
I20260806 21:11:13.682694 19027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:13.661584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50110 (local address 127.27.225.253:43287)
0806 21:11:13.661707 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.661714 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.673585 (+ 11871us) server_negotiation.cc:407] Connection header received
0806 21:11:13.673796 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.673799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.673879 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.673942 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.674810 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.675584 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.676688 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:13.677033 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:13.677104 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:13.678765 (+ 1661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:13.678786 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:13.678789 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:13.678817 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:13.680786 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.681271 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.681276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.681278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.681372 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:13.681676 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:13.681679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:13.681681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:13.681940 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:13.682044 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:13.682401 (+ 357us) server_negotiation.cc:299] Negotiation successful
0806 21:11:13.682530 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 21:11:13.682967 19158 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:11:13.686388 19158 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:11:13.706482 19072 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 95a627c752f54dc4b58866e3d22ddaa8 [term 1 FOLLOWER]: Refusing update from remote peer 23bbb0bb53b4488d836024f02e1d8dae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:11:13.709830 19155 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a627c752f54dc4b58866e3d22ddaa8" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:11:13.715273 19122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95a627c752f54dc4b58866e3d22ddaa8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23bbb0bb53b4488d836024f02e1d8dae. Latest consensus state: current_term: 1 leader_uuid: "23bbb0bb53b4488d836024f02e1d8dae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23bbb0bb53b4488d836024f02e1d8dae" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42013 } } peers { permanent_uuid: "247dedc1aff84f0587ee62e2097743e5" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 } } peers { permanent_uuid: "95a627c752f54dc4b58866e3d22ddaa8" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } } }
I20260806 21:11:13.715453 19122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95a627c752f54dc4b58866e3d22ddaa8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:11:13.716845 18996 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 247dedc1aff84f0587ee62e2097743e5 [term 1 FOLLOWER]: Refusing update from remote peer 23bbb0bb53b4488d836024f02e1d8dae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:11:13.717586 19155 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae [LEADER]: Connected to new peer: Peer: permanent_uuid: "247dedc1aff84f0587ee62e2097743e5" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:11:13.725250 19106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 247dedc1aff84f0587ee62e2097743e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23bbb0bb53b4488d836024f02e1d8dae. Latest consensus state: current_term: 1 leader_uuid: "23bbb0bb53b4488d836024f02e1d8dae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23bbb0bb53b4488d836024f02e1d8dae" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42013 } } peers { permanent_uuid: "247dedc1aff84f0587ee62e2097743e5" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 } } peers { permanent_uuid: "95a627c752f54dc4b58866e3d22ddaa8" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } } }
I20260806 21:11:13.725438 19106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 247dedc1aff84f0587ee62e2097743e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:11:13.729593 19155 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "23bbb0bb53b4488d836024f02e1d8dae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23bbb0bb53b4488d836024f02e1d8dae" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42013 } } peers { permanent_uuid: "247dedc1aff84f0587ee62e2097743e5" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 } } peers { permanent_uuid: "95a627c752f54dc4b58866e3d22ddaa8" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } } }
I20260806 21:11:13.729781 19155 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:13.731309 19106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 247dedc1aff84f0587ee62e2097743e5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "23bbb0bb53b4488d836024f02e1d8dae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23bbb0bb53b4488d836024f02e1d8dae" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42013 } } peers { permanent_uuid: "247dedc1aff84f0587ee62e2097743e5" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 } } peers { permanent_uuid: "95a627c752f54dc4b58866e3d22ddaa8" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } } }
I20260806 21:11:13.731462 19106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 247dedc1aff84f0587ee62e2097743e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:11:13.731930 19160 mvcc.cc:205] Tried to move back new op lower bound from 7315663559491235840 to 7315663559304732672. Current Snapshot: MvccSnapshot[applied={T|T < 7315663559491235840}]
I20260806 21:11:13.733938 19163 mvcc.cc:205] Tried to move back new op lower bound from 7315663559491235840 to 7315663559304732672. Current Snapshot: MvccSnapshot[applied={T|T < 7315663559491235840}]
I20260806 21:11:13.736426 19137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "23bbb0bb53b4488d836024f02e1d8dae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23bbb0bb53b4488d836024f02e1d8dae" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42013 } } peers { permanent_uuid: "247dedc1aff84f0587ee62e2097743e5" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 } } peers { permanent_uuid: "95a627c752f54dc4b58866e3d22ddaa8" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } } }
I20260806 21:11:13.736608 19137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:13.737716 19158 catalog_manager.cc:1347] Generated new cluster ID: 5fb437b4db56475392a1bf0088ec3b67
I20260806 21:11:13.737798 19158 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:11:13.740314 19122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95a627c752f54dc4b58866e3d22ddaa8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "23bbb0bb53b4488d836024f02e1d8dae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23bbb0bb53b4488d836024f02e1d8dae" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 42013 } } peers { permanent_uuid: "247dedc1aff84f0587ee62e2097743e5" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 43287 } } peers { permanent_uuid: "95a627c752f54dc4b58866e3d22ddaa8" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 42185 } } }
I20260806 21:11:13.740461 19122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95a627c752f54dc4b58866e3d22ddaa8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:11:13.742357 19161 mvcc.cc:205] Tried to move back new op lower bound from 7315663559491235840 to 7315663559304732672. Current Snapshot: MvccSnapshot[applied={T|T < 7315663559491235840}]
I20260806 21:11:13.770996 19158 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:11:13.772439 19158 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:11:13.788556 19158 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23bbb0bb53b4488d836024f02e1d8dae: Generated new TSK 0
I20260806 21:11:13.789494 19158 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 21:11:13.826378 18948 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:13.822713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42864 (local address 127.27.225.254:42013)
0806 21:11:13.822856 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.822862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.824989 (+ 2127us) server_negotiation.cc:407] Connection header received
0806 21:11:13.825243 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.825247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.825307 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.825376 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.826221 (+ 845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42864: BlockingRecv error: recv got EOF from 127.0.0.1:42864 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 21:11:13.827893 19104 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:13.823579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37354 (local address 127.27.225.252:42185)
0806 21:11:13.823709 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.823714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.826811 (+ 3097us) server_negotiation.cc:407] Connection header received
0806 21:11:13.827003 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.827006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.827057 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.827142 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.827764 (+ 622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37354: BlockingRecv error: recv got EOF from 127.0.0.1:37354 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 21:11:13.829810 19027 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:13.824282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50114 (local address 127.27.225.253:43287)
0806 21:11:13.824409 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.824414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.828478 (+ 4064us) server_negotiation.cc:407] Connection header received
0806 21:11:13.828948 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.828952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.829009 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.829066 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.829683 (+ 617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50114: BlockingRecv error: recv got EOF from 127.0.0.1:50114 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260806 21:11:13.834892 19027 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:13.833597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50124 (local address 127.27.225.253:43287)
0806 21:11:13.833729 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.833733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.833794 (+ 61us) server_negotiation.cc:407] Connection header received
0806 21:11:13.834073 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.834076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.834167 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.834226 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.834762 (+ 536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50124: BlockingRecv error: recv got EOF from 127.0.0.1:50124 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260806 21:11:13.840547 19027 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:13.838874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50140 (local address 127.27.225.253:43287)
0806 21:11:13.838989 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.838994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.839558 (+ 564us) server_negotiation.cc:407] Connection header received
0806 21:11:13.839754 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.839758 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.839822 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.839882 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.840431 (+ 549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50140: BlockingRecv error: recv got EOF from 127.0.0.1:50140 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260806 21:11:13.847206 19027 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:13.845153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50146 (local address 127.27.225.253:43287)
0806 21:11:13.845293 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.845298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.845940 (+ 642us) server_negotiation.cc:407] Connection header received
0806 21:11:13.846207 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.846210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.846269 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.846325 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.847052 (+ 727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50146: BlockingRecv error: recv got EOF from 127.0.0.1:50146 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 21:11:13.855792 19027 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:13.853802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50148 (local address 127.27.225.253:43287)
0806 21:11:13.854014 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.854021 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.854655 (+ 634us) server_negotiation.cc:407] Connection header received
0806 21:11:13.854860 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.854865 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.854920 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.854972 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.855668 (+ 696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50148: BlockingRecv error: recv got EOF from 127.0.0.1:50148 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
W20260806 21:11:13.868470 19027 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:13.866541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50154 (local address 127.27.225.253:43287)
0806 21:11:13.866687 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.866692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.867334 (+ 642us) server_negotiation.cc:407] Connection header received
0806 21:11:13.867559 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.867563 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.867625 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.867684 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.868342 (+ 658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50154: BlockingRecv error: recv got EOF from 127.0.0.1:50154 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260806 21:11:13.888228 19027 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:13.886929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50160 (local address 127.27.225.253:43287)
0806 21:11:13.887079 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.887084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.887205 (+ 121us) server_negotiation.cc:407] Connection header received
0806 21:11:13.887402 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.887407 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.887464 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.887562 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.888066 (+ 504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50160: BlockingRecv error: recv got EOF from 127.0.0.1:50160 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260806 21:11:13.924394 19027 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:13.922717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50174 (local address 127.27.225.253:43287)
0806 21:11:13.922901 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.922906 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.923053 (+ 147us) server_negotiation.cc:407] Connection header received
0806 21:11:13.923122 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.923124 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.923222 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.923369 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.923993 (+ 624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50174: BlockingRecv error: recv got EOF from 127.0.0.1:50174 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260806 21:11:13.991870 19027 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:13.990784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50178 (local address 127.27.225.253:43287)
0806 21:11:13.990952 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:13.990957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:13.991063 (+ 106us) server_negotiation.cc:407] Connection header received
0806 21:11:13.991189 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:13.991192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:13.991241 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:13.991347 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:13.991766 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50178: BlockingRecv error: recv got EOF from 127.0.0.1:50178 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260806 21:11:14.123205 19027 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:14.122112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50190 (local address 127.27.225.253:43287)
0806 21:11:14.122253 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:14.122257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:14.122394 (+ 137us) server_negotiation.cc:407] Connection header received
0806 21:11:14.122501 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:14.122504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:14.122552 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:14.122637 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:14.123096 (+ 459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50190: BlockingRecv error: recv got EOF from 127.0.0.1:50190 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 21:11:14.382905 19027 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:14.381903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50200 (local address 127.27.225.253:43287)
0806 21:11:14.382067 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:14.382071 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:14.382093 (+ 22us) server_negotiation.cc:407] Connection header received
0806 21:11:14.382156 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:14.382158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:14.382211 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:14.382308 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:14.382780 (+ 472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50200: BlockingRecv error: recv got EOF from 127.0.0.1:50200 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 21:11:14.632287 19118 catalog_manager.cc:1259] Loaded cluster ID: 5fb437b4db56475392a1bf0088ec3b67
I20260806 21:11:14.632421 19118 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 247dedc1aff84f0587ee62e2097743e5: loading cluster ID for follower catalog manager: success
I20260806 21:11:14.634383 19118 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 247dedc1aff84f0587ee62e2097743e5: acquiring CA information for follower catalog manager: success
I20260806 21:11:14.635780 19118 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 247dedc1aff84f0587ee62e2097743e5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 21:11:14.645625 19133 catalog_manager.cc:1259] Loaded cluster ID: 5fb437b4db56475392a1bf0088ec3b67
I20260806 21:11:14.645725 19133 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 95a627c752f54dc4b58866e3d22ddaa8: loading cluster ID for follower catalog manager: success
I20260806 21:11:14.647665 19133 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 95a627c752f54dc4b58866e3d22ddaa8: acquiring CA information for follower catalog manager: success
I20260806 21:11:14.648909 19133 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 95a627c752f54dc4b58866e3d22ddaa8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 21:11:14.901400 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:14.897230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50216 (local address 127.27.225.253:43287)
0806 21:11:14.897530 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:14.897537 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:14.897557 (+ 20us) server_negotiation.cc:407] Connection header received
0806 21:11:14.897627 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:14.897630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:14.897759 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:14.897918 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:11:14.898230 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:14.898777 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:14.899742 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:14.899984 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:14.900105 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:14.900466 (+ 361us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:11:14.900931 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:11:14.901075 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:14.901139 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 21:11:14.901209 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":92,"threads_started":1}
I20260806 21:11:14.911213 19242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:14.905920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37366 (local address 127.27.225.252:42185)
0806 21:11:14.906232 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:14.906238 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:14.906363 (+ 125us) server_negotiation.cc:407] Connection header received
0806 21:11:14.906735 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:14.906739 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:14.906879 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:14.907021 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:11:14.907296 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:14.907850 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:14.909326 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:14.909612 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:14.909852 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:14.910246 (+ 394us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:11:14.910733 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:11:14.910885 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:14.910950 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 21:11:14.911021 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":77,"threads_started":1}
I20260806 21:11:14.920310 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:14.915790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50228 (local address 127.27.225.253:43287)
0806 21:11:14.915968 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:14.915975 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:14.916091 (+ 116us) server_negotiation.cc:407] Connection header received
0806 21:11:14.916305 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:14.916312 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:14.916381 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:14.916437 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:11:14.916902 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:14.917748 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:14.918924 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:14.919175 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:14.919306 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:14.919726 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:11:14.919810 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:11:14.919925 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:14.920071 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 21:11:14.920141 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 21:11:14.921020 19242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:14.916480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37372 (local address 127.27.225.252:42185)
0806 21:11:14.916650 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:14.916655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:14.916669 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:11:14.916720 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:14.916723 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:14.916776 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:14.916852 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:11:14.917448 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:14.918225 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:14.919519 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:14.919804 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:14.919913 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:14.920335 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:11:14.920541 (+ 206us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:11:14.920616 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:14.920821 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 21:11:14.920891 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 21:11:14.922132 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:14.915764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42876 (local address 127.27.225.254:42013)
0806 21:11:14.916520 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:14.916524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:14.916541 (+ 17us) server_negotiation.cc:407] Connection header received
0806 21:11:14.916652 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:14.916655 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:14.916840 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:14.917119 (+ 279us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:11:14.917266 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:14.918172 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:14.919999 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:14.920415 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:14.920838 (+ 423us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:14.921006 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:11:14.921421 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:11:14.921522 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:14.921925 (+ 403us) server_negotiation.cc:299] Negotiation successful
0806 21:11:14.922011 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":91,"threads_started":1}
I20260806 21:11:14.926100 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 18875
I20260806 21:11:14.935345 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 18951
I20260806 21:11:14.942502 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 19029
2026-08-06T21:11:14Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3297 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:11:15 dist-test-slave-69rg krb5kdc[19253](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:11:15 dist-test-slave-69rg krb5kdc[19253](info): set up 2 sockets
Aug 06 21:11:15 dist-test-slave-69rg krb5kdc[19253](info): commencing operation
krb5kdc: starting...
W20260806 21:11:16.016925 28551 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/krb5kdc/test-user.keytab.
Aug 06 21:11:16 dist-test-slave-69rg krb5kdc[19253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:11:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:11:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.225.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
Entry for principal kudu/127.27.225.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab.
I20260806 21:11:16.182539 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:44097
--webserver_interface=127.27.225.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:34271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:44097,127.27.225.253:41155,127.27.225.252:39045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:16.307057 19267 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 21:11:16.307492 19267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:16.307581 19267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:16.307672 19267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:16.312378 19267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:16.312520 19267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:16.312561 19267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:16.312601 19267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:16.312634 19267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:11:16.318604 19267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:34271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:44097,127.27.225.253:41155,127.27.225.252:39045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.254.keytab
--principal=kudu/127.27.225.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.254:44097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.225.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:16.319859 19267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:16.320953 19267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:16.330067 19273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:16.330082 19276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:16.330067 19274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:16.330307 19267 server_base.cc:1034] running on GCE node
I20260806 21:11:16.332217 19267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:16.333508 19267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:16.334827 19267 hybrid_clock.cc:648] HybridClock initialized: now 1786050676334804 us; error 65 us; skew 500 ppm
Aug 06 21:11:16 dist-test-slave-69rg krb5kdc[19253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050676, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:16.338963 19267 init.cc:377] Logged in from keytab as kudu/127.27.225.254@KRBTEST.COM (short username kudu)
I20260806 21:11:16.340193 19267 webserver.cc:460] Webserver started at http://127.27.225.254:41061/ using document root <none> and password file <none>
I20260806 21:11:16.340839 19267 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:16.340911 19267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:16.341180 19267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:16.343621 19267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-0/data/instance:
uuid: "d0e805aa34ca473c8dfe943b7fcf9969"
format_stamp: "Formatted at 2026-08-06 21:11:16 on dist-test-slave-69rg"
server_key: "2bf156d69e641e6df06834f52c69f754"
server_key_iv: "b905b4a8e2b6befad74a613d648099ee"
server_key_version: "encryptionkey@0"
I20260806 21:11:16.344319 19267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-0/wal/instance:
uuid: "d0e805aa34ca473c8dfe943b7fcf9969"
format_stamp: "Formatted at 2026-08-06 21:11:16 on dist-test-slave-69rg"
server_key: "2bf156d69e641e6df06834f52c69f754"
server_key_iv: "b905b4a8e2b6befad74a613d648099ee"
server_key_version: "encryptionkey@0"
I20260806 21:11:16.348765 19267 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260806 21:11:16.351730 19283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:16.353051 19267 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 21:11:16.353224 19267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-0/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-0/wal
uuid: "d0e805aa34ca473c8dfe943b7fcf9969"
format_stamp: "Formatted at 2026-08-06 21:11:16 on dist-test-slave-69rg"
server_key: "2bf156d69e641e6df06834f52c69f754"
server_key_iv: "b905b4a8e2b6befad74a613d648099ee"
server_key_version: "encryptionkey@0"
I20260806 21:11:16.353367 19267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:16.368934 19267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:16.373099 19267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:16.373345 19267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:16.381156 19267 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.254:44097
I20260806 21:11:16.381564 19338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.254:44097 every 8 connection(s)
I20260806 21:11:16.382268 19267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-0/data/info.pb
I20260806 21:11:16.385366 19339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:16.389654 19339 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44097 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:11:16.391911 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 19267
I20260806 21:11:16.392055 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-0/wal/instance
I20260806 21:11:16.392412 28551 external_mini_cluster.cc:1428] Setting key 01db7cfcb44e3447da421edf0643dd7e
Aug 06 21:11:16 dist-test-slave-69rg krb5kdc[19253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050676, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.225.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.253@KRBTEST.COM" created.
I20260806 21:11:16.406849 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.392353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:44097 (local address 127.0.0.1:34022)
0806 21:11:16.393046 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.393084 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:16.394041 (+ 957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:16.394751 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:16.394757 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:16.395215 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:16.395469 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:16.395483 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.396640 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.396643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:16.397420 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:16.397429 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.398205 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.398209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:16.398312 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:16.399021 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:16.399037 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:16.400916 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:16.403195 (+ 2279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:16.403204 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:16.403222 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:16.403477 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:16.403912 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:16.403914 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:16.403916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:16.404060 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:16.404844 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:16.404850 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:16.404986 (+ 136us) client_negotiation.cc:769] Sending connection context
0806 21:11:16.405281 (+ 295us) client_negotiation.cc:240] Negotiation successful
0806 21:11:16.405777 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":194,"thread_start_us":223,"threads_started":1}
I20260806 21:11:16.406849 19342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.392352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34022 (local address 127.27.225.254:44097)
0806 21:11:16.392898 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:16.392919 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:16.393197 (+ 278us) server_negotiation.cc:407] Connection header received
0806 21:11:16.394353 (+ 1156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:16.394357 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:16.394550 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:16.394748 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:16.395621 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.396507 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.397543 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.398026 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.398218 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:16.401091 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:16.401097 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:16.401109 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:16.401130 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:16.403064 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.403659 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.403665 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.403667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.403793 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.404195 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.404198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.404200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.404709 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:16.404855 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:16.405348 (+ 493us) server_negotiation.cc:299] Negotiation successful
0806 21:11:16.405726 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":382,"spinlock_wait_cycles":43904,"thread_start_us":204,"threads_started":1}
I20260806 21:11:16.409605 19339 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.253" port: 41155 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:11:16.410482 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.410197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:41155 (local address 127.0.0.1:57990)
0806 21:11:16.410344 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.410415 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.253:41155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 21:11:16.410746 19284 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.225.253:41155: connect: Connection refused (error 111)
I20260806 21:11:16.411640 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.411435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:41155 (local address 127.0.0.1:58004)
0806 21:11:16.411544 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.411586 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.253:41155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 21:11:16.411887 19339 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.253:41155: Network error: Client connection negotiation failed: client connection to 127.27.225.253:41155: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.253.keytab.
Entry for principal kudu/127.27.225.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.253.keytab.
I20260806 21:11:16.425251 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.253:41155
--webserver_interface=127.27.225.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.253.keytab
--principal=kudu/127.27.225.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:34271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:44097,127.27.225.253:41155,127.27.225.252:39045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:11:16.461174 19339 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.253" port: 41155 } attempt: 1
I20260806 21:11:16.462198 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.461857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:41155 (local address 127.0.0.1:58008)
0806 21:11:16.462034 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.462107 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.253:41155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 21:11:16.463022 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.462788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:41155 (local address 127.0.0.1:58012)
0806 21:11:16.462906 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.462961 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.253:41155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 21:11:16.463231 19339 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.253:41155: Network error: Client connection negotiation failed: client connection to 127.27.225.253:41155: connect: Connection refused (error 111)
I20260806 21:11:16.531431 19339 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.253" port: 41155 } attempt: 2
I20260806 21:11:16.532295 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.531987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:41155 (local address 127.0.0.1:58028)
0806 21:11:16.532147 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.532217 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.253:41155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 21:11:16.533017 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.532798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:41155 (local address 127.0.0.1:58032)
0806 21:11:16.532910 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.532961 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.253:41155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 21:11:16.533149 19339 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.253:41155: Network error: Client connection negotiation failed: client connection to 127.27.225.253:41155: connect: Connection refused (error 111)
W20260806 21:11:16.542593 19346 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 21:11:16.542982 19346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:16.543049 19346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:16.543162 19346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:16.547448 19346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:16.547560 19346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:16.547596 19346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:16.547638 19346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:16.547673 19346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:11:16.553588 19346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:34271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:44097,127.27.225.253:41155,127.27.225.252:39045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.253.keytab
--principal=kudu/127.27.225.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.253:41155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.225.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:16.554850 19346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:16.555930 19346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:16.565562 19355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:16.565562 19352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:16.565580 19353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:16.565793 19346 server_base.cc:1034] running on GCE node
I20260806 21:11:16.566627 19346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:16.567835 19346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:16.569034 19346 hybrid_clock.cc:648] HybridClock initialized: now 1786050676569020 us; error 31 us; skew 500 ppm
Aug 06 21:11:16 dist-test-slave-69rg krb5kdc[19253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050676, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:16.582072 19346 init.cc:377] Logged in from keytab as kudu/127.27.225.253@KRBTEST.COM (short username kudu)
I20260806 21:11:16.583214 19346 webserver.cc:460] Webserver started at http://127.27.225.253:35979/ using document root <none> and password file <none>
I20260806 21:11:16.583779 19346 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:16.583846 19346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:16.584025 19346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:16.586280 19346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-1/data/instance:
uuid: "c6eb45ba87d949408e8cb6aa3b9bf7dc"
format_stamp: "Formatted at 2026-08-06 21:11:16 on dist-test-slave-69rg"
server_key: "ce395b028c27e9be249f649f629abd79"
server_key_iv: "7815e5d03c4c55680f0f605ba80be64e"
server_key_version: "encryptionkey@0"
I20260806 21:11:16.586836 19346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-1/wal/instance:
uuid: "c6eb45ba87d949408e8cb6aa3b9bf7dc"
format_stamp: "Formatted at 2026-08-06 21:11:16 on dist-test-slave-69rg"
server_key: "ce395b028c27e9be249f649f629abd79"
server_key_iv: "7815e5d03c4c55680f0f605ba80be64e"
server_key_version: "encryptionkey@0"
I20260806 21:11:16.590940 19346 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 21:11:16.593490 19363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:16.594828 19346 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:11:16.594965 19346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-1/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-1/wal
uuid: "c6eb45ba87d949408e8cb6aa3b9bf7dc"
format_stamp: "Formatted at 2026-08-06 21:11:16 on dist-test-slave-69rg"
server_key: "ce395b028c27e9be249f649f629abd79"
server_key_iv: "7815e5d03c4c55680f0f605ba80be64e"
server_key_version: "encryptionkey@0"
I20260806 21:11:16.595082 19346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:16.604809 19346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:16.608234 19346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:16.608466 19346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:16.615810 19346 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.253:41155
I20260806 21:11:16.616209 19419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.253:41155 every 8 connection(s)
I20260806 21:11:16.616863 19346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-1/data/info.pb
I20260806 21:11:16.619872 19420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:16.623477 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 19346
I20260806 21:11:16.623653 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-1/wal/instance
I20260806 21:11:16.623967 28551 external_mini_cluster.cc:1428] Setting key e4137128a60dc3940eb54eb548b09753
I20260806 21:11:16.623795 19420 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44097 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:11:16.624315 19339 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.253" port: 41155 } attempt: 3
Aug 06 21:11:16 dist-test-slave-69rg krb5kdc[19253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050676, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for kudu/127.27.225.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.225.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.225.252@KRBTEST.COM" created.
Aug 06 21:11:16 dist-test-slave-69rg krb5kdc[19253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050676, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.253@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:11:16.638545 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.624847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:41155 (local address 127.0.0.1:58036)
0806 21:11:16.625057 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.625074 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:16.625181 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:16.628129 (+ 2948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:16.628135 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:16.628373 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:16.628534 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:16.628544 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.630591 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.630595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:16.631303 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:16.631313 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.631944 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.631948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:16.632167 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:16.632726 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:16.632741 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:16.634220 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:16.636361 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:16.636364 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:16.636366 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:16.636571 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:16.636934 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:16.636936 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:16.636937 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:16.637018 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:16.637920 (+ 902us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:16.637923 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:16.638045 (+ 122us) client_negotiation.cc:769] Sending connection context
0806 21:11:16.638183 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 21:11:16.638360 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":62}
I20260806 21:11:16.639618 19422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.625686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58036 (local address 127.27.225.253:41155)
0806 21:11:16.626486 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:16.626499 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:16.626546 (+ 47us) server_negotiation.cc:407] Connection header received
0806 21:11:16.627612 (+ 1066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:16.627641 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:16.627910 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:16.628706 (+ 796us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:16.628871 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.630442 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.631429 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.631815 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.631988 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:16.634445 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:16.634452 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:16.634468 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:16.634495 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:16.636247 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.636680 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.636692 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.636697 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.636844 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.637110 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.637115 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.637117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.637744 (+ 627us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:16.637956 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:16.638360 (+ 404us) server_negotiation.cc:299] Negotiation successful
0806 21:11:16.638672 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":658,"thread_start_us":167,"threads_started":1}
I20260806 21:11:16.641916 19339 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:11:16.642699 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.642429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:39045 (local address 127.0.0.1:38848)
0806 21:11:16.642558 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.642622 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 21:11:16.643489 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.643232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:39045 (local address 127.0.0.1:38860)
0806 21:11:16.643377 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.643428 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260806 21:11:16.643662 19339 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.252:39045: Network error: Client connection negotiation failed: client connection to 127.27.225.252:39045: connect: Connection refused (error 111)
I20260806 21:11:16.643832 19342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.625897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34024 (local address 127.27.225.254:44097)
0806 21:11:16.626037 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:16.626042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:16.626435 (+ 393us) server_negotiation.cc:407] Connection header received
0806 21:11:16.627857 (+ 1422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:16.627860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:16.627927 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:16.628021 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:16.629097 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.629658 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.631066 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.632275 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.632396 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:16.638858 (+ 6462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:16.638864 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:16.638870 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:16.638902 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:16.641442 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.642106 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.642112 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.642117 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.642213 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.642585 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.642591 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.642596 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.642876 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:16.642977 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:16.643491 (+ 514us) server_negotiation.cc:299] Negotiation successful
0806 21:11:16.643625 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":52}
I20260806 21:11:16.643918 19423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.625856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:44097 (local address 127.0.0.1:34024)
0806 21:11:16.626291 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.626342 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:16.627622 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:16.628075 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:16.628083 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:16.628450 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:16.628951 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:16.628961 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.629809 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.629812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:16.630911 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:16.630922 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.632773 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.632780 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:16.632904 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:16.633972 (+ 1068us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:16.633993 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:16.638506 (+ 4513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:16.641577 (+ 3071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:16.641591 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:16.641599 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:16.641977 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:16.642337 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:16.642343 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:16.642348 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:16.642458 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:16.643056 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:16.643066 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:16.643220 (+ 154us) client_negotiation.cc:769] Sending connection context
0806 21:11:16.643407 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 21:11:16.643635 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":137,"thread_start_us":129,"threads_started":1}
I20260806 21:11:16.645107 19420 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.253" port: 41155 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 21:11:16 dist-test-slave-69rg krb5kdc[19253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050676, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.253@KRBTEST.COM for kudu/127.27.225.253@KRBTEST.COM
I20260806 21:11:16.654251 19423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.645571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:41155 (local address 127.0.0.1:58052)
0806 21:11:16.645787 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.645798 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:16.645917 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:16.646177 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:16.646179 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:16.646317 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:16.646405 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:16.646409 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.647510 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.647513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:16.648042 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:16.648050 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.648565 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.648571 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:16.648692 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:16.649137 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:16.649152 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:16.650752 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:16.652626 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:16.652628 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:16.652630 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:16.652822 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:16.653154 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:16.653157 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:16.653159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:16.653252 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:16.653666 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:16.653669 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:16.653808 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 21:11:16.653958 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 21:11:16.654083 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156}
I20260806 21:11:16.654266 19422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.645688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58052 (local address 127.27.225.253:41155)
0806 21:11:16.645854 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:16.645859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:16.645881 (+ 22us) server_negotiation.cc:407] Connection header received
0806 21:11:16.646015 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:16.646018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:16.646067 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:16.646157 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:16.646520 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.647379 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.648154 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.648387 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.648527 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:16.650893 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:16.650896 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:16.650898 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:16.650919 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:16.652490 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.652912 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.652914 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.652915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.653022 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.653351 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.653353 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.653354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.653540 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:16.653617 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:16.654004 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 21:11:16.654142 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":45}
I20260806 21:11:16.655241 19420 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:11:16.655974 19423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.655729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:39045 (local address 127.0.0.1:38868)
0806 21:11:16.655840 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.655919 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 21:11:16.656679 19365 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.225.252:39045: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.225.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.252.keytab.
Entry for principal kudu/127.27.225.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.252.keytab.
I20260806 21:11:16.657719 19423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.657498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:39045 (local address 127.0.0.1:38876)
0806 21:11:16.657611 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.657664 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 21:11:16.657943 19420 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.252:39045: Network error: Client connection negotiation failed: client connection to 127.27.225.252:39045: connect: Connection refused (error 111)
I20260806 21:11:16.658229 28551 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDiceBW/build/debug/bin/kudu
/tmp/dist-test-taskDiceBW/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.252:39045
--webserver_interface=127.27.225.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.252.keytab
--principal=kudu/127.27.225.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.225.212:34271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.225.254:44097,127.27.225.253:41155,127.27.225.252:39045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:11:16.674845 19339 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } attempt: 1
I20260806 21:11:16.675829 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.675490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:39045 (local address 127.0.0.1:38892)
0806 21:11:16.675676 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.675746 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 21:11:16.676713 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.676440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:39045 (local address 127.0.0.1:38898)
0806 21:11:16.676590 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.676651 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260806 21:11:16.676873 19339 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.252:39045: Network error: Client connection negotiation failed: client connection to 127.27.225.252:39045: connect: Connection refused (error 111)
I20260806 21:11:16.707244 19420 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } attempt: 1
I20260806 21:11:16.708334 19423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.707994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:39045 (local address 127.0.0.1:38914)
0806 21:11:16.708155 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.708218 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 21:11:16.709048 19423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.708832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:39045 (local address 127.0.0.1:38930)
0806 21:11:16.708941 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.708992 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 21:11:16.709201 19420 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.252:39045: Network error: Client connection negotiation failed: client connection to 127.27.225.252:39045: connect: Connection refused (error 111)
I20260806 21:11:16.752058 19339 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } attempt: 2
I20260806 21:11:16.753029 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.752708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:39045 (local address 127.0.0.1:38940)
0806 21:11:16.752881 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.752947 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 21:11:16.753968 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.753671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:39045 (local address 127.0.0.1:38956)
0806 21:11:16.753800 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.753912 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 21:11:16.754150 19339 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.252:39045: Network error: Client connection negotiation failed: client connection to 127.27.225.252:39045: connect: Connection refused (error 111)
W20260806 21:11:16.776247 19427 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 21:11:16.776643 19427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:16.776710 19427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:16.776788 19427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 21:11:16.777390 19420 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } attempt: 2
I20260806 21:11:16.778311 19423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.777959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:39045 (local address 127.0.0.1:38972)
0806 21:11:16.778146 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.778218 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 21:11:16.779083 19423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.778853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:39045 (local address 127.0.0.1:38986)
0806 21:11:16.778967 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.779018 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.225.252:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 21:11:16.779230 19420 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.225.252:39045: Network error: Client connection negotiation failed: client connection to 127.27.225.252:39045: connect: Connection refused (error 111)
W20260806 21:11:16.781543 19427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:16.781611 19427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:16.781632 19427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:16.781687 19427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:16.781713 19427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:11:16.787604 19427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.225.212:34271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.27.225.254:44097,127.27.225.253:41155,127.27.225.252:39045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/krb5kdc/kudu_127.27.225.252.keytab
--principal=kudu/127.27.225.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.225.252:39045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.225.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:16.788776 19427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:16.789906 19427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:16.798341 19433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:16.798605 19435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:16.798606 19432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:16.799085 19427 server_base.cc:1034] running on GCE node
I20260806 21:11:16.799726 19427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:16.800844 19427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:16.802057 19427 hybrid_clock.cc:648] HybridClock initialized: now 1786050676802027 us; error 44 us; skew 500 ppm
Aug 06 21:11:16 dist-test-slave-69rg krb5kdc[19253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050676, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:16.804903 19427 init.cc:377] Logged in from keytab as kudu/127.27.225.252@KRBTEST.COM (short username kudu)
I20260806 21:11:16.805914 19427 webserver.cc:460] Webserver started at http://127.27.225.252:40527/ using document root <none> and password file <none>
I20260806 21:11:16.806435 19427 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:16.806493 19427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:16.806660 19427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:16.808651 19427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-2/data/instance:
uuid: "e444cfc6b41d418d871899c082b4c02d"
format_stamp: "Formatted at 2026-08-06 21:11:16 on dist-test-slave-69rg"
server_key: "63c5b3a6a376aa5c861d4dffa5a3415b"
server_key_iv: "b45c3a72b38fc23a7f633add2ebfbf19"
server_key_version: "encryptionkey@0"
I20260806 21:11:16.809175 19427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-2/wal/instance:
uuid: "e444cfc6b41d418d871899c082b4c02d"
format_stamp: "Formatted at 2026-08-06 21:11:16 on dist-test-slave-69rg"
server_key: "63c5b3a6a376aa5c861d4dffa5a3415b"
server_key_iv: "b45c3a72b38fc23a7f633add2ebfbf19"
server_key_version: "encryptionkey@0"
I20260806 21:11:16.812911 19427 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260806 21:11:16.815382 19442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:16.816676 19427 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 21:11:16.816833 19427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-2/data,/tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-2/wal
uuid: "e444cfc6b41d418d871899c082b4c02d"
format_stamp: "Formatted at 2026-08-06 21:11:16 on dist-test-slave-69rg"
server_key: "63c5b3a6a376aa5c861d4dffa5a3415b"
server_key_iv: "b45c3a72b38fc23a7f633add2ebfbf19"
server_key_version: "encryptionkey@0"
I20260806 21:11:16.816957 19427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:16.835824 19427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:16.839064 19427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:16.839284 19427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:16.846437 19427 rpc_server.cc:307] RPC server started. Bound to: 127.27.225.252:39045
I20260806 21:11:16.846748 19495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.225.252:39045 every 8 connection(s)
I20260806 21:11:16.847420 19427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-2/data/info.pb
I20260806 21:11:16.850139 19496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:16.853330 19339 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } attempt: 3
I20260806 21:11:16.854162 19496 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44097 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:11:16.856451 28551 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDiceBW/build/debug/bin/kudu as pid 19427
I20260806 21:11:16.856600 28551 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDiceBW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050378084738-28551-0/minicluster-data/master-2/wal/instance
I20260806 21:11:16.856930 28551 external_mini_cluster.cc:1428] Setting key 49ef998c895c8076ac3767d58f896b71
Aug 06 21:11:16 dist-test-slave-69rg krb5kdc[19253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050676, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.254@KRBTEST.COM for kudu/127.27.225.252@KRBTEST.COM
Aug 06 21:11:16 dist-test-slave-69rg krb5kdc[19253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050676, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.252@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
Aug 06 21:11:16 dist-test-slave-69rg krb5kdc[19253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.252@KRBTEST.COM
I20260806 21:11:16.867795 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.853933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:39045 (local address 127.0.0.1:38996)
0806 21:11:16.854106 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.854124 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:16.854230 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:16.856671 (+ 2441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:16.856674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:16.856914 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:16.857035 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:16.857043 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.859042 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.859046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:16.859618 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:16.859626 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.860611 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.860615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:16.860706 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:16.861298 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:16.861311 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:16.862887 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:16.865630 (+ 2743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:16.865635 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:16.865641 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:16.866001 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:16.866381 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:16.866385 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:16.866387 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:16.866484 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:16.867205 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:16.867212 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:16.867348 (+ 136us) client_negotiation.cc:769] Sending connection context
0806 21:11:16.867467 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 21:11:16.867621 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":3}
I20260806 21:11:16.868201 19342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.856019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34040 (local address 127.27.225.254:44097)
0806 21:11:16.856144 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:16.856148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:16.856484 (+ 336us) server_negotiation.cc:407] Connection header received
0806 21:11:16.856673 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:16.856676 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:16.856726 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:16.856781 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:16.858088 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.858887 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.860350 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.860623 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.860714 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:16.864261 (+ 3547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:16.864264 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:16.864266 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:16.864294 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:16.865933 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.866588 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.866594 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.866595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.866686 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.867033 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.867037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.867038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.867386 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:16.867493 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:16.867919 (+ 426us) server_negotiation.cc:299] Negotiation successful
0806 21:11:16.868050 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":238,"server-negotiator.queue_time_us":50}
I20260806 21:11:16.869261 19500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.855951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.254:44097 (local address 127.0.0.1:34040)
0806 21:11:16.856353 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.856402 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:16.856567 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:16.856987 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:16.856993 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:16.857484 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:16.857949 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:16.857963 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.859021 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.859028 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:16.860228 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:16.860236 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.860809 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.860816 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:16.860961 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:16.861506 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:16.861529 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:16.864084 (+ 2555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:16.866061 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:16.866073 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:16.866086 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:16.866459 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:16.866807 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:16.866810 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:16.866812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:16.866925 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:16.867549 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:16.867556 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:16.867681 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 21:11:16.868355 (+ 674us) client_negotiation.cc:240] Negotiation successful
0806 21:11:16.868552 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":89,"spinlock_wait_cycles":5504,"thread_start_us":169,"threads_started":1}
I20260806 21:11:16.869261 19501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.857515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38998 (local address 127.27.225.252:39045)
0806 21:11:16.857793 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:16.857804 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:16.857994 (+ 190us) server_negotiation.cc:407] Connection header received
0806 21:11:16.858143 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:16.858146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:16.858190 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:16.858275 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:16.859160 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.859909 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.861054 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.861446 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.861544 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:16.864271 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:16.864275 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:16.864278 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:16.864342 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:16.866847 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.867358 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.867362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.867363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.867447 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.867766 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.867769 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.867771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.867981 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:16.868138 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:16.868775 (+ 637us) server_negotiation.cc:299] Negotiation successful
0806 21:11:16.868893 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"spinlock_wait_cycles":1792,"thread_start_us":101,"threads_started":1}
I20260806 21:11:16.869261 19499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.854711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38996 (local address 127.27.225.252:39045)
0806 21:11:16.855152 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:16.855167 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:16.855201 (+ 34us) server_negotiation.cc:407] Connection header received
0806 21:11:16.855901 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:16.855924 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:16.856347 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:16.857226 (+ 879us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:16.857757 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.858925 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.859858 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.860485 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.860641 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:16.863053 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:16.863065 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:16.863082 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:16.863110 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:16.865495 (+ 2385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.866104 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.866114 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.866119 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.866269 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.866588 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.866591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.866593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.867076 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:16.867217 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:16.867640 (+ 423us) server_negotiation.cc:299] Negotiation successful
0806 21:11:16.868222 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":107,"server-negotiator.queue_time_us":312,"thread_start_us":143,"threads_started":1}
I20260806 21:11:16.870361 19420 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } attempt: 3
I20260806 21:11:16.870666 19496 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.253" port: 41155 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:11:16.874018 19339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969: Bootstrap starting.
Aug 06 21:11:16 dist-test-slave-69rg krb5kdc[19253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050676, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.253@KRBTEST.COM for kudu/127.27.225.252@KRBTEST.COM
Aug 06 21:11:16 dist-test-slave-69rg krb5kdc[19253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050676, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.252@KRBTEST.COM for kudu/127.27.225.253@KRBTEST.COM
I20260806 21:11:16.877959 19339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969: Neither blocks nor log segments found. Creating new log.
Aug 06 21:11:16 dist-test-slave-69rg krb5kdc[19253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.254@KRBTEST.COM
I20260806 21:11:16.879083 19339 log.cc:824] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:16.881412 19423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.870837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:39045 (local address 127.0.0.1:39008)
0806 21:11:16.871041 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.871055 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:16.871134 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:16.871625 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:16.871628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:16.871828 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:16.871949 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:16.871959 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.873001 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.873006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:16.873747 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:16.873758 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.874381 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.874384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:16.874466 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:16.874982 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:16.874996 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:16.876734 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:16.879561 (+ 2827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:16.879564 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:16.879566 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:16.880090 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:16.880403 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:16.880406 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:16.880408 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:16.880496 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:16.880939 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:16.880942 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:16.881032 (+ 90us) client_negotiation.cc:769] Sending connection context
0806 21:11:16.881114 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 21:11:16.881251 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":3}
I20260806 21:11:16.881443 19339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969: No bootstrap required, opened a new log
I20260806 21:11:16.881760 19500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.871093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.253:41155 (local address 127.0.0.1:58068)
0806 21:11:16.871641 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.871656 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:16.871775 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:16.872133 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:16.872135 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:16.872354 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:16.872470 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:16.872478 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.873510 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.873514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:16.874367 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:16.874378 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.875391 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.875395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:16.875499 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:16.876054 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:16.876068 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:16.877659 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:16.879705 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:16.879708 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:16.879710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:16.879959 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:16.880342 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:16.880345 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:16.880347 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:16.880433 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:16.880933 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:16.880938 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:16.881040 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 21:11:16.881439 (+ 399us) client_negotiation.cc:240] Negotiation successful
0806 21:11:16.881580 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 21:11:16.881786 19422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.871310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58068 (local address 127.27.225.253:41155)
0806 21:11:16.871402 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:16.871408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:16.871708 (+ 300us) server_negotiation.cc:407] Connection header received
0806 21:11:16.871881 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:16.871885 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:16.871940 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:16.871995 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:16.872606 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.873369 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.874506 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.875155 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.875241 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:16.877868 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:16.877873 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:16.877876 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:16.877902 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:16.879575 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.880097 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.880104 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.880108 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.880200 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.880550 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.880553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.880556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.880782 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:16.880855 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:16.881246 (+ 391us) server_negotiation.cc:299] Negotiation successful
0806 21:11:16.881383 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 21:11:16.882284 19501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.871301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39008 (local address 127.27.225.252:39045)
0806 21:11:16.871394 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:16.871399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:16.871410 (+ 11us) server_negotiation.cc:407] Connection header received
0806 21:11:16.871446 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:16.871450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:16.871494 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:16.872068 (+ 574us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:16.872160 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.872867 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.873933 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.874246 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.874683 (+ 437us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:16.876889 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:16.876893 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:16.876896 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:16.876920 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:16.879208 (+ 2288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.880216 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.880219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.880221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.880301 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.880601 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.880604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.880605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.880821 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:16.881968 (+ 1147us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:16.882044 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 21:11:16.882152 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 21:11:16.882701 19342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.872307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34056 (local address 127.27.225.254:44097)
0806 21:11:16.872411 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:16.872415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:16.872581 (+ 166us) server_negotiation.cc:407] Connection header received
0806 21:11:16.872747 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:16.872750 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:16.872805 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:16.872854 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:16.873674 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.874433 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.875559 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.875926 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.876021 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:16.879047 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:16.879051 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:16.879053 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:16.879087 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:16.880752 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.881257 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.881260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.881261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.881348 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.881670 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.881673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.881675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.881953 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:16.882106 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:16.882455 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 21:11:16.882565 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 21:11:16.885345 19420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6eb45ba87d949408e8cb6aa3b9bf7dc: Bootstrap starting.
I20260806 21:11:16.885576 19339 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44097 } } peers { permanent_uuid: "c6eb45ba87d949408e8cb6aa3b9bf7dc" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 41155 } } peers { permanent_uuid: "e444cfc6b41d418d871899c082b4c02d" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } }
I20260806 21:11:16.887144 19496 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:11:16.885795 19339 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e805aa34ca473c8dfe943b7fcf9969, State: Initialized, Role: FOLLOWER
I20260806 21:11:16.887874 19339 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44097 } } peers { permanent_uuid: "c6eb45ba87d949408e8cb6aa3b9bf7dc" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 41155 } } peers { permanent_uuid: "e444cfc6b41d418d871899c082b4c02d" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } }
I20260806 21:11:16.888504 19420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6eb45ba87d949408e8cb6aa3b9bf7dc: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:16.888607 19504 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44097 } } peers { permanent_uuid: "c6eb45ba87d949408e8cb6aa3b9bf7dc" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 41155 } } peers { permanent_uuid: "e444cfc6b41d418d871899c082b4c02d" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } } }
I20260806 21:11:16.888803 19504 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:11:16.889367 19420 log.cc:824] T 00000000000000000000000000000000 P c6eb45ba87d949408e8cb6aa3b9bf7dc: Log is configured to *not* fsync() on all Append() calls
Aug 06 21:11:16 dist-test-slave-69rg krb5kdc[19253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.225.253@KRBTEST.COM
I20260806 21:11:16.889726 19339 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969 [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 21:11:16 dist-test-slave-69rg krb5kdc[19253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050676, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.225.252@KRBTEST.COM for kudu/127.27.225.252@KRBTEST.COM
I20260806 21:11:16.894682 19422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.883983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58076 (local address 127.27.225.253:41155)
0806 21:11:16.884082 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:16.884086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:16.884238 (+ 152us) server_negotiation.cc:407] Connection header received
0806 21:11:16.884356 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:16.884359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:16.884400 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:16.884489 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:16.885070 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.885556 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.886471 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.886809 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.886925 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:16.890381 (+ 3456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:16.890385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:16.890388 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:16.890413 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:16.892170 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.892681 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.892684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.892686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.892774 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.893134 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.893137 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.893139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.893367 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:16.893563 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:16.894378 (+ 815us) server_negotiation.cc:299] Negotiation successful
0806 21:11:16.894498 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 21:11:16.897183 19420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6eb45ba87d949408e8cb6aa3b9bf7dc: No bootstrap required, opened a new log
I20260806 21:11:16.898355 19500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.887623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.225.252:39045 (local address 127.0.0.1:39020)
0806 21:11:16.887754 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:16.887769 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:16.887854 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:16.889445 (+ 1591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:16.889448 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:16.889619 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:16.889723 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:16.889731 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.890786 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.890789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:16.891604 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:16.891618 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.892227 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.892233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:16.892327 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:16.892771 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:16.892786 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:16.894421 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:16.896819 (+ 2398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:16.896826 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:16.896828 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:16.897023 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:16.897295 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:16.897300 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:16.897305 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:16.897389 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:16.897747 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:16.897750 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:16.897904 (+ 154us) client_negotiation.cc:769] Sending connection context
0806 21:11:16.898014 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 21:11:16.898175 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 21:11:16.898936 19501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.887833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39020 (local address 127.27.225.252:39045)
0806 21:11:16.889208 (+ 1375us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:16.889214 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:16.889233 (+ 19us) server_negotiation.cc:407] Connection header received
0806 21:11:16.889270 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:16.889273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:16.889325 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:16.889867 (+ 542us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:16.889948 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.890674 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.891745 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.892102 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.892539 (+ 437us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:16.894552 (+ 2013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:16.894556 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:16.894559 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:16.894579 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:16.896705 (+ 2126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.897119 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.897125 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.897126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.897208 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.897479 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.897482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.897484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.897656 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:16.897759 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:16.898690 (+ 931us) server_negotiation.cc:299] Negotiation successful
0806 21:11:16.898809 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1289}
W20260806 21:11:16.898941 19516 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 21:11:16.899194 28551 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 21:11:16.899312 19516 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:11:16.901531 19420 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6eb45ba87d949408e8cb6aa3b9bf7dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44097 } } peers { permanent_uuid: "c6eb45ba87d949408e8cb6aa3b9bf7dc" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 41155 } } peers { permanent_uuid: "e444cfc6b41d418d871899c082b4c02d" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } }
I20260806 21:11:16.901800 19420 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6eb45ba87d949408e8cb6aa3b9bf7dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6eb45ba87d949408e8cb6aa3b9bf7dc, State: Initialized, Role: FOLLOWER
I20260806 21:11:16.902187 19420 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6eb45ba87d949408e8cb6aa3b9bf7dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44097 } } peers { permanent_uuid: "c6eb45ba87d949408e8cb6aa3b9bf7dc" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 41155 } } peers { permanent_uuid: "e444cfc6b41d418d871899c082b4c02d" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } }
I20260806 21:11:16.902633 19496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e444cfc6b41d418d871899c082b4c02d: Bootstrap starting.
I20260806 21:11:16.902793 19518 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6eb45ba87d949408e8cb6aa3b9bf7dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44097 } } peers { permanent_uuid: "c6eb45ba87d949408e8cb6aa3b9bf7dc" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 41155 } } peers { permanent_uuid: "e444cfc6b41d418d871899c082b4c02d" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } } }
I20260806 21:11:16.902997 19518 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6eb45ba87d949408e8cb6aa3b9bf7dc [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:11:16.903551 19420 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6eb45ba87d949408e8cb6aa3b9bf7dc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:16.905493 19496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e444cfc6b41d418d871899c082b4c02d: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:16.906457 19311 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d0e805aa34ca473c8dfe943b7fcf9969"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34056
I20260806 21:11:16.906740 19311 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 21:11:16.906869 19311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:16.908828 19311 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44097 } } peers { permanent_uuid: "c6eb45ba87d949408e8cb6aa3b9bf7dc" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 41155 } } peers { permanent_uuid: "e444cfc6b41d418d871899c082b4c02d" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } }
W20260806 21:11:16.909053 19529 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c6eb45ba87d949408e8cb6aa3b9bf7dc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:11:16.909160 19529 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:11:16.894841 19267 master_runner.cc:186] Error getting master registration for 127.27.225.253:41155: OK, instance_id { permanent_uuid: "c6eb45ba87d949408e8cb6aa3b9bf7dc" instance_seqno: 1786050676613786 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 21:11:16.911403 19267 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 21:11:16.911527 19392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c6eb45ba87d949408e8cb6aa3b9bf7dc"
I20260806 21:11:16.911777 19392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6eb45ba87d949408e8cb6aa3b9bf7dc [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:16.912293 19496 log.cc:824] T 00000000000000000000000000000000 P e444cfc6b41d418d871899c082b4c02d: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:16.913205 19392 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c6eb45ba87d949408e8cb6aa3b9bf7dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e805aa34ca473c8dfe943b7fcf9969 in term 1.
I20260806 21:11:16.914000 19284 leader_election.cc:302] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6eb45ba87d949408e8cb6aa3b9bf7dc, d0e805aa34ca473c8dfe943b7fcf9969; no voters:
I20260806 21:11:16.914399 19504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:16.914660 19504 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969 [term 1 LEADER]: Becoming Leader. State: Replica: d0e805aa34ca473c8dfe943b7fcf9969, State: Running, Role: LEADER
I20260806 21:11:16.915011 19504 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44097 } } peers { permanent_uuid: "c6eb45ba87d949408e8cb6aa3b9bf7dc" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 41155 } } peers { permanent_uuid: "e444cfc6b41d418d871899c082b4c02d" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } }
I20260806 21:11:16.921481 19311 leader_election.cc:288] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969 [CANDIDATE]: Term 1 election: Requested vote from peers c6eb45ba87d949408e8cb6aa3b9bf7dc (127.27.225.253:41155), e444cfc6b41d418d871899c082b4c02d (127.27.225.252:39045)
I20260806 21:11:16.922302 19532 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0e805aa34ca473c8dfe943b7fcf9969. Latest consensus state: current_term: 1 leader_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44097 } } peers { permanent_uuid: "c6eb45ba87d949408e8cb6aa3b9bf7dc" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 41155 } } peers { permanent_uuid: "e444cfc6b41d418d871899c082b4c02d" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } } }
I20260806 21:11:16.922631 19532 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969 [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:16.919775 19346 master_runner.cc:186] Error getting master registration for 127.27.225.252:39045: OK, instance_id { permanent_uuid: "e444cfc6b41d418d871899c082b4c02d" instance_seqno: 1786050676844423 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 21:11:16.923873 19346 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 21:11:16.925329 19496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e444cfc6b41d418d871899c082b4c02d: No bootstrap required, opened a new log
I20260806 21:11:16.927205 19534 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:11:16.928176 19534 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:11:16.931335 19496 raft_consensus.cc:348] T 00000000000000000000000000000000 P e444cfc6b41d418d871899c082b4c02d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44097 } } peers { permanent_uuid: "c6eb45ba87d949408e8cb6aa3b9bf7dc" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 41155 } } peers { permanent_uuid: "e444cfc6b41d418d871899c082b4c02d" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } }
I20260806 21:11:16.931847 19496 raft_consensus.cc:729] T 00000000000000000000000000000000 P e444cfc6b41d418d871899c082b4c02d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e444cfc6b41d418d871899c082b4c02d, State: Initialized, Role: FOLLOWER
I20260806 21:11:16.932422 19496 consensus_queue.cc:260] T 00000000000000000000000000000000 P e444cfc6b41d418d871899c082b4c02d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44097 } } peers { permanent_uuid: "c6eb45ba87d949408e8cb6aa3b9bf7dc" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 41155 } } peers { permanent_uuid: "e444cfc6b41d418d871899c082b4c02d" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } }
I20260806 21:11:16.934754 19469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e444cfc6b41d418d871899c082b4c02d"
I20260806 21:11:16.935005 19469 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e444cfc6b41d418d871899c082b4c02d [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:16.936852 19469 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e444cfc6b41d418d871899c082b4c02d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0e805aa34ca473c8dfe943b7fcf9969 in term 1.
I20260806 21:11:16.937561 19496 sys_catalog.cc:564] T 00000000000000000000000000000000 P e444cfc6b41d418d871899c082b4c02d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:16.941974 19543 sys_catalog.cc:455] T 00000000000000000000000000000000 P e444cfc6b41d418d871899c082b4c02d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44097 } } peers { permanent_uuid: "c6eb45ba87d949408e8cb6aa3b9bf7dc" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 41155 } } peers { permanent_uuid: "e444cfc6b41d418d871899c082b4c02d" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } } }
I20260806 21:11:16.942116 19543 sys_catalog.cc:458] T 00000000000000000000000000000000 P e444cfc6b41d418d871899c082b4c02d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:11:16.942147 19469 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e444cfc6b41d418d871899c082b4c02d [term 1 FOLLOWER]: Refusing update from remote peer d0e805aa34ca473c8dfe943b7fcf9969: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:11:16.942986 19532 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e444cfc6b41d418d871899c082b4c02d" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:11:16.947373 19543 sys_catalog.cc:455] T 00000000000000000000000000000000 P e444cfc6b41d418d871899c082b4c02d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0e805aa34ca473c8dfe943b7fcf9969. Latest consensus state: current_term: 1 leader_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44097 } } peers { permanent_uuid: "c6eb45ba87d949408e8cb6aa3b9bf7dc" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 41155 } } peers { permanent_uuid: "e444cfc6b41d418d871899c082b4c02d" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } } }
I20260806 21:11:16.947538 19543 sys_catalog.cc:458] T 00000000000000000000000000000000 P e444cfc6b41d418d871899c082b4c02d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:11:16.948127 19392 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c6eb45ba87d949408e8cb6aa3b9bf7dc [term 1 FOLLOWER]: Refusing update from remote peer d0e805aa34ca473c8dfe943b7fcf9969: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:11:16.949024 19532 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6eb45ba87d949408e8cb6aa3b9bf7dc" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 41155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:11:16.952265 19501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.926845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39030 (local address 127.27.225.252:39045)
0806 21:11:16.926973 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:16.926978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:16.932646 (+ 5668us) server_negotiation.cc:407] Connection header received
0806 21:11:16.932746 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:16.932751 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:16.932806 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:16.932902 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:16.937914 (+ 5012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.938684 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.940781 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.941101 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.941176 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:16.948512 (+ 7336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:16.948517 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:16.948519 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:16.948549 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:16.950270 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.950959 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.950962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.950964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.951044 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.951321 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.951325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.951326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.951526 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:16.951626 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:16.951990 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 21:11:16.952110 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 21:11:16.956254 19518 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6eb45ba87d949408e8cb6aa3b9bf7dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0e805aa34ca473c8dfe943b7fcf9969. Latest consensus state: current_term: 1 leader_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44097 } } peers { permanent_uuid: "c6eb45ba87d949408e8cb6aa3b9bf7dc" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 41155 } } peers { permanent_uuid: "e444cfc6b41d418d871899c082b4c02d" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } } }
I20260806 21:11:16.956414 19518 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6eb45ba87d949408e8cb6aa3b9bf7dc [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:11:16.957266 19342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.926087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34066 (local address 127.27.225.254:44097)
0806 21:11:16.926211 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:16.926217 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:16.931174 (+ 4957us) server_negotiation.cc:407] Connection header received
0806 21:11:16.931377 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:16.931381 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:16.931431 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:16.931482 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:16.932793 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.933594 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.937331 (+ 3737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.937685 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.937762 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:16.941027 (+ 3265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:16.941030 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:16.941033 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:16.941061 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:16.949437 (+ 8376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.949989 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.949993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.949995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.950089 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.950393 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.950396 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.950398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.950627 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:16.950747 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:16.956916 (+ 6169us) server_negotiation.cc:299] Negotiation successful
0806 21:11:16.957098 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 21:11:16.957412 19554 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e444cfc6b41d418d871899c082b4c02d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:11:16.957491 19554 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:11:16.960361 19422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:16.930578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58092 (local address 127.27.225.253:41155)
0806 21:11:16.930689 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:16.930693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:16.938541 (+ 7848us) server_negotiation.cc:407] Connection header received
0806 21:11:16.938741 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:16.938745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:16.938802 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:16.938853 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:16.939981 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.940726 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.953976 (+ 13250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:16.954333 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:16.954408 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:16.956139 (+ 1731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:16.956143 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:16.956146 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:16.956175 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:16.958439 (+ 2264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.958992 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.958996 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.958998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.959091 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:16.959388 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:16.959391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:16.959393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:16.959622 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:16.959724 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:16.960086 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 21:11:16.960207 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 21:11:16.963501 19532 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44097 } } peers { permanent_uuid: "c6eb45ba87d949408e8cb6aa3b9bf7dc" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 41155 } } peers { permanent_uuid: "e444cfc6b41d418d871899c082b4c02d" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } } }
I20260806 21:11:16.963642 19532 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969 [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:16.964839 19543 sys_catalog.cc:455] T 00000000000000000000000000000000 P e444cfc6b41d418d871899c082b4c02d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44097 } } peers { permanent_uuid: "c6eb45ba87d949408e8cb6aa3b9bf7dc" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 41155 } } peers { permanent_uuid: "e444cfc6b41d418d871899c082b4c02d" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } } }
I20260806 21:11:16.964979 19543 sys_catalog.cc:458] T 00000000000000000000000000000000 P e444cfc6b41d418d871899c082b4c02d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:11:16.965231 19541 mvcc.cc:205] Tried to move back new op lower bound from 7315663572731650048 to 7315663572647514112. Current Snapshot: MvccSnapshot[applied={T|T < 7315663572731650048}]
I20260806 21:11:16.966490 19504 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44097 } } peers { permanent_uuid: "c6eb45ba87d949408e8cb6aa3b9bf7dc" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 41155 } } peers { permanent_uuid: "e444cfc6b41d418d871899c082b4c02d" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } } }
I20260806 21:11:16.966614 19504 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969 [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:16.969766 19518 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6eb45ba87d949408e8cb6aa3b9bf7dc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e805aa34ca473c8dfe943b7fcf9969" member_type: VOTER last_known_addr { host: "127.27.225.254" port: 44097 } } peers { permanent_uuid: "c6eb45ba87d949408e8cb6aa3b9bf7dc" member_type: VOTER last_known_addr { host: "127.27.225.253" port: 41155 } } peers { permanent_uuid: "e444cfc6b41d418d871899c082b4c02d" member_type: VOTER last_known_addr { host: "127.27.225.252" port: 39045 } } }
I20260806 21:11:16.969921 19518 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6eb45ba87d949408e8cb6aa3b9bf7dc [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:11:16.970544 19558 mvcc.cc:205] Tried to move back new op lower bound from 7315663572731650048 to 7315663572647514112. Current Snapshot: MvccSnapshot[applied={T|T < 7315663572731650048}]
I20260806 21:11:16.975225 19534 catalog_manager.cc:1347] Generated new cluster ID: 3d5d970fcecc4529af86ae552fd105fe
I20260806 21:11:16.975310 19534 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:11:17.000773 19534 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:11:17.004550 19534 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:11:17.027841 19534 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0e805aa34ca473c8dfe943b7fcf9969: Generated new TSK 0
I20260806 21:11:17.028854 19534 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 21:11:17.061146 19342 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:17.057553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34074 (local address 127.27.225.254:44097)
0806 21:11:17.057697 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:17.057703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:17.059942 (+ 2239us) server_negotiation.cc:407] Connection header received
0806 21:11:17.060161 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:17.060164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:17.060219 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:17.060274 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:17.061001 (+ 727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34074: BlockingRecv error: recv got EOF from 127.0.0.1:34074 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 21:11:17.062731 19501 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:17.058550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39040 (local address 127.27.225.252:39045)
0806 21:11:17.058669 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:17.058675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:17.061686 (+ 3011us) server_negotiation.cc:407] Connection header received
0806 21:11:17.061918 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:17.061922 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:17.061978 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:17.062035 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:17.062599 (+ 564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39040: BlockingRecv error: recv got EOF from 127.0.0.1:39040 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 21:11:17.064375 19422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:17.059288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58108 (local address 127.27.225.253:41155)
0806 21:11:17.059410 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:17.059414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:17.063150 (+ 3736us) server_negotiation.cc:407] Connection header received
0806 21:11:17.063568 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:17.063572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:17.063626 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:17.063679 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:17.064253 (+ 574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58108: BlockingRecv error: recv got EOF from 127.0.0.1:58108 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 21:11:17.069361 19422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:17.067740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58124 (local address 127.27.225.253:41155)
0806 21:11:17.067855 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:17.067859 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:17.068309 (+ 450us) server_negotiation.cc:407] Connection header received
0806 21:11:17.068542 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:17.068545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:17.068627 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:17.068683 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:17.069234 (+ 551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58124: BlockingRecv error: recv got EOF from 127.0.0.1:58124 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 21:11:17.074746 19422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:17.073147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58132 (local address 127.27.225.253:41155)
0806 21:11:17.073250 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:17.073254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:17.073742 (+ 488us) server_negotiation.cc:407] Connection header received
0806 21:11:17.073969 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:17.073973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:17.074029 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:17.074082 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:17.074627 (+ 545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58132: BlockingRecv error: recv got EOF from 127.0.0.1:58132 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 21:11:17.081138 19422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:17.079370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58134 (local address 127.27.225.253:41155)
0806 21:11:17.079500 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:17.079504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:17.080083 (+ 579us) server_negotiation.cc:407] Connection header received
0806 21:11:17.080311 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:17.080314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:17.080371 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:17.080435 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:17.081016 (+ 581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58134: BlockingRecv error: recv got EOF from 127.0.0.1:58134 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 21:11:17.090350 19422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:17.088445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58138 (local address 127.27.225.253:41155)
0806 21:11:17.088615 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:17.088620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:17.089193 (+ 573us) server_negotiation.cc:407] Connection header received
0806 21:11:17.089442 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:17.089446 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:17.089506 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:17.089566 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:17.090215 (+ 649us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58138: BlockingRecv error: recv got EOF from 127.0.0.1:58138 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260806 21:11:17.103343 19422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:17.101415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58152 (local address 127.27.225.253:41155)
0806 21:11:17.101549 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:17.101554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:17.102176 (+ 622us) server_negotiation.cc:407] Connection header received
0806 21:11:17.102418 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:17.102421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:17.102484 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:17.102545 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:17.103207 (+ 662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58152: BlockingRecv error: recv got EOF from 127.0.0.1:58152 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 21:11:17.124099 19422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:17.122523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58154 (local address 127.27.225.253:41155)
0806 21:11:17.122651 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:17.122656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:17.122923 (+ 267us) server_negotiation.cc:407] Connection header received
0806 21:11:17.123144 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:17.123148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:17.123207 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:17.123263 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:17.123968 (+ 705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58154: BlockingRecv error: recv got EOF from 127.0.0.1:58154 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 21:11:17.160423 19422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:17.159241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58158 (local address 127.27.225.253:41155)
0806 21:11:17.159386 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:17.159391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:17.159567 (+ 176us) server_negotiation.cc:407] Connection header received
0806 21:11:17.159686 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:17.159690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:17.159747 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:17.159850 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:17.160294 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58158: BlockingRecv error: recv got EOF from 127.0.0.1:58158 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 21:11:17.228211 19422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:17.226994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58168 (local address 127.27.225.253:41155)
0806 21:11:17.227160 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:17.227165 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:17.227358 (+ 193us) server_negotiation.cc:407] Connection header received
0806 21:11:17.227483 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:17.227485 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:17.227535 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:17.227610 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:17.228081 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58168: BlockingRecv error: recv got EOF from 127.0.0.1:58168 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 21:11:17.360250 19422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:17.358800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58170 (local address 127.27.225.253:41155)
0806 21:11:17.358940 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:17.358946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:17.359179 (+ 233us) server_negotiation.cc:407] Connection header received
0806 21:11:17.359381 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:17.359385 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:17.359450 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:17.359549 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:17.360083 (+ 534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58170: BlockingRecv error: recv got EOF from 127.0.0.1:58170 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260806 21:11:17.620064 19422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:17.618845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58184 (local address 127.27.225.253:41155)
0806 21:11:17.619006 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:17.619010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:17.619169 (+ 159us) server_negotiation.cc:407] Connection header received
0806 21:11:17.619305 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:17.619308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:17.619355 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:17.619428 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:17.619948 (+ 520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58184: BlockingRecv error: recv got EOF from 127.0.0.1:58184 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 21:11:17.910620 19529 catalog_manager.cc:1259] Loaded cluster ID: 3d5d970fcecc4529af86ae552fd105fe
I20260806 21:11:17.910720 19529 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c6eb45ba87d949408e8cb6aa3b9bf7dc: loading cluster ID for follower catalog manager: success
I20260806 21:11:17.912478 19529 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c6eb45ba87d949408e8cb6aa3b9bf7dc: acquiring CA information for follower catalog manager: success
I20260806 21:11:17.913730 19529 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c6eb45ba87d949408e8cb6aa3b9bf7dc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 21:11:17.959012 19554 catalog_manager.cc:1259] Loaded cluster ID: 3d5d970fcecc4529af86ae552fd105fe
I20260806 21:11:17.959141 19554 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e444cfc6b41d418d871899c082b4c02d: loading cluster ID for follower catalog manager: success
I20260806 21:11:17.961098 19554 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e444cfc6b41d418d871899c082b4c02d: acquiring CA information for follower catalog manager: success
I20260806 21:11:17.962541 19554 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e444cfc6b41d418d871899c082b4c02d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 21:11:18.139415 19632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:18.134387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58196 (local address 127.27.225.253:41155)
0806 21:11:18.134677 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:18.134685 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:18.134811 (+ 126us) server_negotiation.cc:407] Connection header received
0806 21:11:18.135010 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:18.135021 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:18.135171 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:18.135307 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:11:18.135607 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:18.136192 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:18.137199 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:18.137485 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:18.137626 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:18.138028 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:11:18.138643 (+ 615us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:11:18.139107 (+ 464us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:18.139149 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 21:11:18.139228 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":82,"threads_started":1}
I20260806 21:11:18.147022 19640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:18.142870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39042 (local address 127.27.225.252:39045)
0806 21:11:18.143155 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:18.143159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:18.143176 (+ 17us) server_negotiation.cc:407] Connection header received
0806 21:11:18.143263 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:18.143265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:18.143395 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:18.143505 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:11:18.143761 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:18.144320 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:18.145249 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:18.145510 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:18.145626 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:18.146086 (+ 460us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:11:18.146537 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:11:18.146662 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:18.146743 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 21:11:18.146801 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":87,"threads_started":1}
I20260806 21:11:18.154435 19640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:18.150341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39056 (local address 127.27.225.252:39045)
0806 21:11:18.150464 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:18.150471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:18.150635 (+ 164us) server_negotiation.cc:407] Connection header received
0806 21:11:18.150768 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:18.150772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:18.150826 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:18.150932 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:11:18.151458 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:18.152192 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:18.153112 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:18.153336 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:18.153446 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:18.153883 (+ 437us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:11:18.153973 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:11:18.154078 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:18.154217 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 21:11:18.154285 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 21:11:18.154950 19632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:18.150449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58204 (local address 127.27.225.253:41155)
0806 21:11:18.150587 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:18.150594 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:18.150763 (+ 169us) server_negotiation.cc:407] Connection header received
0806 21:11:18.150954 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:18.150957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:18.151015 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:18.151103 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:11:18.151348 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:18.152172 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:18.153567 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:18.153934 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:18.154021 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:18.154394 (+ 373us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:11:18.154490 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:11:18.154546 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:18.154742 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 21:11:18.154822 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 21:11:18.155850 19647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:18.150159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34084 (local address 127.27.225.254:44097)
0806 21:11:18.150637 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:18.150642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:18.151023 (+ 381us) server_negotiation.cc:407] Connection header received
0806 21:11:18.151128 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:18.151130 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:18.151245 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:18.151371 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:11:18.151662 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:18.152200 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:18.154115 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:18.154483 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:18.154616 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:18.154958 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:11:18.155305 (+ 347us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:11:18.155421 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:18.155665 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 21:11:18.155738 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":77,"threads_started":1}
I20260806 21:11:18.159323 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 19267
I20260806 21:11:18.167367 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 19346
I20260806 21:11:18.174140 28551 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDiceBW/build/debug/bin/kudu with pid 19427
2026-08-06T21:11:18Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3231 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6529 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (300047 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST